COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM...

47
COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ Shaharuddin Salleh Universiti Teknologi Malaysia Skudai, Johor, Malaysia Albert Y. Zomaya University of Sydney Sydney, New South Wales, Australia Sakhinah Abu Bakar National University of Malaysia Bangi, Selangor, Malaysia

Transcript of COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM...

Page 1: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 17 2007 2123 Char Count= 0

COMPUTING FORNUMERICAL METHODSUSING VISUAL C++

Shaharuddin SallehUniversiti Teknologi MalaysiaSkudai Johor Malaysia

Albert Y ZomayaUniversity of SydneySydney New South Wales Australia

Sakhinah Abu BakarNational University of MalaysiaBangi Selangor Malaysia

iii

Innodata
9780470192627jpg

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

i

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

COMPUTING FORNUMERICAL METHODSUSING VISUAL C++

i

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

ii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 17 2007 2123 Char Count= 0

COMPUTING FORNUMERICAL METHODSUSING VISUAL C++

Shaharuddin SallehUniversiti Teknologi MalaysiaSkudai Johor Malaysia

Albert Y ZomayaUniversity of SydneySydney New South Wales Australia

Sakhinah Abu BakarNational University of MalaysiaBangi Selangor Malaysia

iii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

Copyright Ccopy 2008 by John Wiley amp Sons Inc All rights reserved

Published by John Wiley amp Sons Inc Hoboken New JerseyPublished simultaneously in Canada

No part of this publication may be reproduced stored in a retrieval system or transmitted in any form orby any means electronic mechanical photocopying recording scanning or otherwise except aspermitted under Section 107 or 108 of the 1976 United States Copyright Act without either the priorwritten permission of the Publisher or authorization through payment of the appropriate per-copy fee tothe Copyright Clearance Center Inc 222 Rosewood Drive Danvers MA 01923 (978) 750-8400 fax(978) 750-4470 or on the web at wwwcopyrightcom Requests to the Publisher for permission shouldbe addressed to the Permissions Department John Wiley amp Sons Inc 111 River Street HobokenNJ 07030 (201) 748-6011 fax (201) 748-6008 or online at httpwwwwileycomgopermission

Limit of LiabilityDisclaimer of Warranty While the publisher and author have used their bestefforts in preparing this book they make no representations or warranties with respect to the accuracyor completeness of the contents of this book and specifically disclaim any implied warranties ofmerchantability or fitness for a particular purpose No warranty may be created or extended by salesrepresentatives or written sales materials The advice and strategies contained herein may not be suitablefor your situation You should consult with a professional where appropriate Neither the publisher norauthor shall be liable for any loss of profit or any other commercial damages including but not limited tospecial incidental consequential or other damages

For general information on our other products and services or for technical support please contact ourCustomer Care Department within the United States at (800) 762-2974 outside the United States at(317) 572-3993 or fax (317) 572-4002

Wiley also publishes its books in a variety of electronic formats Some content that appears in print maynot be available in electronic formats For more information about Wiley products visit our web site atwwwwileycom

Wiley Bicentennial Logo Richard J Pacifico

Library of Congress Cataloging-in-Publication Data

Salleh Shaharuddin 1956-Computing for numerical methods using Visual c++ by Shaharuddin Salleh Albert YZomaya Sakhinah Abu Bakar

p cm ndash (Wiley series on parallel and distributed computing)Includes indexISBN 978-0-470-12795-7 (cloth)1 Microsoft Visual C NET 2 Numerical analysisndashData processing

I Zomaya Albert Y II Bakar Sakhinah Abu 1982ndash III Title

QA7673C154S28 20070052prime768ndashdc22

2007023224Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

iv

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

To our familiesfor their help courage support and patience

v

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

vi

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

TRADEMARKS

Microsoft is the trademark of Microsoft Corporation Redmond WAVisual Studio and Visual StudioNet are the trademarks of Microsoft CorporationVisual C++ and Visual C++Net are the trademarks of Microsoft CorporationMicrosoft Foundation Classes is the trademark of Microsoft CorporationMatlab is the trademark of The Mathworks Inc Natick MAMaple is the trademark of Waterloo Maple Inc Waterloo Ontario CanadaMathematica is the trademark of Wolfram Research Inc Champaign IL

vii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 2: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

i

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

COMPUTING FORNUMERICAL METHODSUSING VISUAL C++

i

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

ii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 17 2007 2123 Char Count= 0

COMPUTING FORNUMERICAL METHODSUSING VISUAL C++

Shaharuddin SallehUniversiti Teknologi MalaysiaSkudai Johor Malaysia

Albert Y ZomayaUniversity of SydneySydney New South Wales Australia

Sakhinah Abu BakarNational University of MalaysiaBangi Selangor Malaysia

iii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

Copyright Ccopy 2008 by John Wiley amp Sons Inc All rights reserved

Published by John Wiley amp Sons Inc Hoboken New JerseyPublished simultaneously in Canada

No part of this publication may be reproduced stored in a retrieval system or transmitted in any form orby any means electronic mechanical photocopying recording scanning or otherwise except aspermitted under Section 107 or 108 of the 1976 United States Copyright Act without either the priorwritten permission of the Publisher or authorization through payment of the appropriate per-copy fee tothe Copyright Clearance Center Inc 222 Rosewood Drive Danvers MA 01923 (978) 750-8400 fax(978) 750-4470 or on the web at wwwcopyrightcom Requests to the Publisher for permission shouldbe addressed to the Permissions Department John Wiley amp Sons Inc 111 River Street HobokenNJ 07030 (201) 748-6011 fax (201) 748-6008 or online at httpwwwwileycomgopermission

Limit of LiabilityDisclaimer of Warranty While the publisher and author have used their bestefforts in preparing this book they make no representations or warranties with respect to the accuracyor completeness of the contents of this book and specifically disclaim any implied warranties ofmerchantability or fitness for a particular purpose No warranty may be created or extended by salesrepresentatives or written sales materials The advice and strategies contained herein may not be suitablefor your situation You should consult with a professional where appropriate Neither the publisher norauthor shall be liable for any loss of profit or any other commercial damages including but not limited tospecial incidental consequential or other damages

For general information on our other products and services or for technical support please contact ourCustomer Care Department within the United States at (800) 762-2974 outside the United States at(317) 572-3993 or fax (317) 572-4002

Wiley also publishes its books in a variety of electronic formats Some content that appears in print maynot be available in electronic formats For more information about Wiley products visit our web site atwwwwileycom

Wiley Bicentennial Logo Richard J Pacifico

Library of Congress Cataloging-in-Publication Data

Salleh Shaharuddin 1956-Computing for numerical methods using Visual c++ by Shaharuddin Salleh Albert YZomaya Sakhinah Abu Bakar

p cm ndash (Wiley series on parallel and distributed computing)Includes indexISBN 978-0-470-12795-7 (cloth)1 Microsoft Visual C NET 2 Numerical analysisndashData processing

I Zomaya Albert Y II Bakar Sakhinah Abu 1982ndash III Title

QA7673C154S28 20070052prime768ndashdc22

2007023224Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

iv

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

To our familiesfor their help courage support and patience

v

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

vi

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

TRADEMARKS

Microsoft is the trademark of Microsoft Corporation Redmond WAVisual Studio and Visual StudioNet are the trademarks of Microsoft CorporationVisual C++ and Visual C++Net are the trademarks of Microsoft CorporationMicrosoft Foundation Classes is the trademark of Microsoft CorporationMatlab is the trademark of The Mathworks Inc Natick MAMaple is the trademark of Waterloo Maple Inc Waterloo Ontario CanadaMathematica is the trademark of Wolfram Research Inc Champaign IL

vii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 3: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

COMPUTING FORNUMERICAL METHODSUSING VISUAL C++

i

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

ii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 17 2007 2123 Char Count= 0

COMPUTING FORNUMERICAL METHODSUSING VISUAL C++

Shaharuddin SallehUniversiti Teknologi MalaysiaSkudai Johor Malaysia

Albert Y ZomayaUniversity of SydneySydney New South Wales Australia

Sakhinah Abu BakarNational University of MalaysiaBangi Selangor Malaysia

iii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

Copyright Ccopy 2008 by John Wiley amp Sons Inc All rights reserved

Published by John Wiley amp Sons Inc Hoboken New JerseyPublished simultaneously in Canada

No part of this publication may be reproduced stored in a retrieval system or transmitted in any form orby any means electronic mechanical photocopying recording scanning or otherwise except aspermitted under Section 107 or 108 of the 1976 United States Copyright Act without either the priorwritten permission of the Publisher or authorization through payment of the appropriate per-copy fee tothe Copyright Clearance Center Inc 222 Rosewood Drive Danvers MA 01923 (978) 750-8400 fax(978) 750-4470 or on the web at wwwcopyrightcom Requests to the Publisher for permission shouldbe addressed to the Permissions Department John Wiley amp Sons Inc 111 River Street HobokenNJ 07030 (201) 748-6011 fax (201) 748-6008 or online at httpwwwwileycomgopermission

Limit of LiabilityDisclaimer of Warranty While the publisher and author have used their bestefforts in preparing this book they make no representations or warranties with respect to the accuracyor completeness of the contents of this book and specifically disclaim any implied warranties ofmerchantability or fitness for a particular purpose No warranty may be created or extended by salesrepresentatives or written sales materials The advice and strategies contained herein may not be suitablefor your situation You should consult with a professional where appropriate Neither the publisher norauthor shall be liable for any loss of profit or any other commercial damages including but not limited tospecial incidental consequential or other damages

For general information on our other products and services or for technical support please contact ourCustomer Care Department within the United States at (800) 762-2974 outside the United States at(317) 572-3993 or fax (317) 572-4002

Wiley also publishes its books in a variety of electronic formats Some content that appears in print maynot be available in electronic formats For more information about Wiley products visit our web site atwwwwileycom

Wiley Bicentennial Logo Richard J Pacifico

Library of Congress Cataloging-in-Publication Data

Salleh Shaharuddin 1956-Computing for numerical methods using Visual c++ by Shaharuddin Salleh Albert YZomaya Sakhinah Abu Bakar

p cm ndash (Wiley series on parallel and distributed computing)Includes indexISBN 978-0-470-12795-7 (cloth)1 Microsoft Visual C NET 2 Numerical analysisndashData processing

I Zomaya Albert Y II Bakar Sakhinah Abu 1982ndash III Title

QA7673C154S28 20070052prime768ndashdc22

2007023224Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

iv

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

To our familiesfor their help courage support and patience

v

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

vi

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

TRADEMARKS

Microsoft is the trademark of Microsoft Corporation Redmond WAVisual Studio and Visual StudioNet are the trademarks of Microsoft CorporationVisual C++ and Visual C++Net are the trademarks of Microsoft CorporationMicrosoft Foundation Classes is the trademark of Microsoft CorporationMatlab is the trademark of The Mathworks Inc Natick MAMaple is the trademark of Waterloo Maple Inc Waterloo Ontario CanadaMathematica is the trademark of Wolfram Research Inc Champaign IL

vii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 4: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

ii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 17 2007 2123 Char Count= 0

COMPUTING FORNUMERICAL METHODSUSING VISUAL C++

Shaharuddin SallehUniversiti Teknologi MalaysiaSkudai Johor Malaysia

Albert Y ZomayaUniversity of SydneySydney New South Wales Australia

Sakhinah Abu BakarNational University of MalaysiaBangi Selangor Malaysia

iii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

Copyright Ccopy 2008 by John Wiley amp Sons Inc All rights reserved

Published by John Wiley amp Sons Inc Hoboken New JerseyPublished simultaneously in Canada

No part of this publication may be reproduced stored in a retrieval system or transmitted in any form orby any means electronic mechanical photocopying recording scanning or otherwise except aspermitted under Section 107 or 108 of the 1976 United States Copyright Act without either the priorwritten permission of the Publisher or authorization through payment of the appropriate per-copy fee tothe Copyright Clearance Center Inc 222 Rosewood Drive Danvers MA 01923 (978) 750-8400 fax(978) 750-4470 or on the web at wwwcopyrightcom Requests to the Publisher for permission shouldbe addressed to the Permissions Department John Wiley amp Sons Inc 111 River Street HobokenNJ 07030 (201) 748-6011 fax (201) 748-6008 or online at httpwwwwileycomgopermission

Limit of LiabilityDisclaimer of Warranty While the publisher and author have used their bestefforts in preparing this book they make no representations or warranties with respect to the accuracyor completeness of the contents of this book and specifically disclaim any implied warranties ofmerchantability or fitness for a particular purpose No warranty may be created or extended by salesrepresentatives or written sales materials The advice and strategies contained herein may not be suitablefor your situation You should consult with a professional where appropriate Neither the publisher norauthor shall be liable for any loss of profit or any other commercial damages including but not limited tospecial incidental consequential or other damages

For general information on our other products and services or for technical support please contact ourCustomer Care Department within the United States at (800) 762-2974 outside the United States at(317) 572-3993 or fax (317) 572-4002

Wiley also publishes its books in a variety of electronic formats Some content that appears in print maynot be available in electronic formats For more information about Wiley products visit our web site atwwwwileycom

Wiley Bicentennial Logo Richard J Pacifico

Library of Congress Cataloging-in-Publication Data

Salleh Shaharuddin 1956-Computing for numerical methods using Visual c++ by Shaharuddin Salleh Albert YZomaya Sakhinah Abu Bakar

p cm ndash (Wiley series on parallel and distributed computing)Includes indexISBN 978-0-470-12795-7 (cloth)1 Microsoft Visual C NET 2 Numerical analysisndashData processing

I Zomaya Albert Y II Bakar Sakhinah Abu 1982ndash III Title

QA7673C154S28 20070052prime768ndashdc22

2007023224Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

iv

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

To our familiesfor their help courage support and patience

v

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

vi

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

TRADEMARKS

Microsoft is the trademark of Microsoft Corporation Redmond WAVisual Studio and Visual StudioNet are the trademarks of Microsoft CorporationVisual C++ and Visual C++Net are the trademarks of Microsoft CorporationMicrosoft Foundation Classes is the trademark of Microsoft CorporationMatlab is the trademark of The Mathworks Inc Natick MAMaple is the trademark of Waterloo Maple Inc Waterloo Ontario CanadaMathematica is the trademark of Wolfram Research Inc Champaign IL

vii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 5: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 17 2007 2123 Char Count= 0

COMPUTING FORNUMERICAL METHODSUSING VISUAL C++

Shaharuddin SallehUniversiti Teknologi MalaysiaSkudai Johor Malaysia

Albert Y ZomayaUniversity of SydneySydney New South Wales Australia

Sakhinah Abu BakarNational University of MalaysiaBangi Selangor Malaysia

iii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

Copyright Ccopy 2008 by John Wiley amp Sons Inc All rights reserved

Published by John Wiley amp Sons Inc Hoboken New JerseyPublished simultaneously in Canada

No part of this publication may be reproduced stored in a retrieval system or transmitted in any form orby any means electronic mechanical photocopying recording scanning or otherwise except aspermitted under Section 107 or 108 of the 1976 United States Copyright Act without either the priorwritten permission of the Publisher or authorization through payment of the appropriate per-copy fee tothe Copyright Clearance Center Inc 222 Rosewood Drive Danvers MA 01923 (978) 750-8400 fax(978) 750-4470 or on the web at wwwcopyrightcom Requests to the Publisher for permission shouldbe addressed to the Permissions Department John Wiley amp Sons Inc 111 River Street HobokenNJ 07030 (201) 748-6011 fax (201) 748-6008 or online at httpwwwwileycomgopermission

Limit of LiabilityDisclaimer of Warranty While the publisher and author have used their bestefforts in preparing this book they make no representations or warranties with respect to the accuracyor completeness of the contents of this book and specifically disclaim any implied warranties ofmerchantability or fitness for a particular purpose No warranty may be created or extended by salesrepresentatives or written sales materials The advice and strategies contained herein may not be suitablefor your situation You should consult with a professional where appropriate Neither the publisher norauthor shall be liable for any loss of profit or any other commercial damages including but not limited tospecial incidental consequential or other damages

For general information on our other products and services or for technical support please contact ourCustomer Care Department within the United States at (800) 762-2974 outside the United States at(317) 572-3993 or fax (317) 572-4002

Wiley also publishes its books in a variety of electronic formats Some content that appears in print maynot be available in electronic formats For more information about Wiley products visit our web site atwwwwileycom

Wiley Bicentennial Logo Richard J Pacifico

Library of Congress Cataloging-in-Publication Data

Salleh Shaharuddin 1956-Computing for numerical methods using Visual c++ by Shaharuddin Salleh Albert YZomaya Sakhinah Abu Bakar

p cm ndash (Wiley series on parallel and distributed computing)Includes indexISBN 978-0-470-12795-7 (cloth)1 Microsoft Visual C NET 2 Numerical analysisndashData processing

I Zomaya Albert Y II Bakar Sakhinah Abu 1982ndash III Title

QA7673C154S28 20070052prime768ndashdc22

2007023224Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

iv

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

To our familiesfor their help courage support and patience

v

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

vi

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

TRADEMARKS

Microsoft is the trademark of Microsoft Corporation Redmond WAVisual Studio and Visual StudioNet are the trademarks of Microsoft CorporationVisual C++ and Visual C++Net are the trademarks of Microsoft CorporationMicrosoft Foundation Classes is the trademark of Microsoft CorporationMatlab is the trademark of The Mathworks Inc Natick MAMaple is the trademark of Waterloo Maple Inc Waterloo Ontario CanadaMathematica is the trademark of Wolfram Research Inc Champaign IL

vii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 6: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

Copyright Ccopy 2008 by John Wiley amp Sons Inc All rights reserved

Published by John Wiley amp Sons Inc Hoboken New JerseyPublished simultaneously in Canada

No part of this publication may be reproduced stored in a retrieval system or transmitted in any form orby any means electronic mechanical photocopying recording scanning or otherwise except aspermitted under Section 107 or 108 of the 1976 United States Copyright Act without either the priorwritten permission of the Publisher or authorization through payment of the appropriate per-copy fee tothe Copyright Clearance Center Inc 222 Rosewood Drive Danvers MA 01923 (978) 750-8400 fax(978) 750-4470 or on the web at wwwcopyrightcom Requests to the Publisher for permission shouldbe addressed to the Permissions Department John Wiley amp Sons Inc 111 River Street HobokenNJ 07030 (201) 748-6011 fax (201) 748-6008 or online at httpwwwwileycomgopermission

Limit of LiabilityDisclaimer of Warranty While the publisher and author have used their bestefforts in preparing this book they make no representations or warranties with respect to the accuracyor completeness of the contents of this book and specifically disclaim any implied warranties ofmerchantability or fitness for a particular purpose No warranty may be created or extended by salesrepresentatives or written sales materials The advice and strategies contained herein may not be suitablefor your situation You should consult with a professional where appropriate Neither the publisher norauthor shall be liable for any loss of profit or any other commercial damages including but not limited tospecial incidental consequential or other damages

For general information on our other products and services or for technical support please contact ourCustomer Care Department within the United States at (800) 762-2974 outside the United States at(317) 572-3993 or fax (317) 572-4002

Wiley also publishes its books in a variety of electronic formats Some content that appears in print maynot be available in electronic formats For more information about Wiley products visit our web site atwwwwileycom

Wiley Bicentennial Logo Richard J Pacifico

Library of Congress Cataloging-in-Publication Data

Salleh Shaharuddin 1956-Computing for numerical methods using Visual c++ by Shaharuddin Salleh Albert YZomaya Sakhinah Abu Bakar

p cm ndash (Wiley series on parallel and distributed computing)Includes indexISBN 978-0-470-12795-7 (cloth)1 Microsoft Visual C NET 2 Numerical analysisndashData processing

I Zomaya Albert Y II Bakar Sakhinah Abu 1982ndash III Title

QA7673C154S28 20070052prime768ndashdc22

2007023224Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

iv

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

To our familiesfor their help courage support and patience

v

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

vi

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

TRADEMARKS

Microsoft is the trademark of Microsoft Corporation Redmond WAVisual Studio and Visual StudioNet are the trademarks of Microsoft CorporationVisual C++ and Visual C++Net are the trademarks of Microsoft CorporationMicrosoft Foundation Classes is the trademark of Microsoft CorporationMatlab is the trademark of The Mathworks Inc Natick MAMaple is the trademark of Waterloo Maple Inc Waterloo Ontario CanadaMathematica is the trademark of Wolfram Research Inc Champaign IL

vii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 7: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

To our familiesfor their help courage support and patience

v

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

vi

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

TRADEMARKS

Microsoft is the trademark of Microsoft Corporation Redmond WAVisual Studio and Visual StudioNet are the trademarks of Microsoft CorporationVisual C++ and Visual C++Net are the trademarks of Microsoft CorporationMicrosoft Foundation Classes is the trademark of Microsoft CorporationMatlab is the trademark of The Mathworks Inc Natick MAMaple is the trademark of Waterloo Maple Inc Waterloo Ontario CanadaMathematica is the trademark of Wolfram Research Inc Champaign IL

vii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 8: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

vi

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

TRADEMARKS

Microsoft is the trademark of Microsoft Corporation Redmond WAVisual Studio and Visual StudioNet are the trademarks of Microsoft CorporationVisual C++ and Visual C++Net are the trademarks of Microsoft CorporationMicrosoft Foundation Classes is the trademark of Microsoft CorporationMatlab is the trademark of The Mathworks Inc Natick MAMaple is the trademark of Waterloo Maple Inc Waterloo Ontario CanadaMathematica is the trademark of Wolfram Research Inc Champaign IL

vii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 9: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

TRADEMARKS

Microsoft is the trademark of Microsoft Corporation Redmond WAVisual Studio and Visual StudioNet are the trademarks of Microsoft CorporationVisual C++ and Visual C++Net are the trademarks of Microsoft CorporationMicrosoft Foundation Classes is the trademark of Microsoft CorporationMatlab is the trademark of The Mathworks Inc Natick MAMaple is the trademark of Waterloo Maple Inc Waterloo Ontario CanadaMathematica is the trademark of Wolfram Research Inc Champaign IL

vii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 10: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

viii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 11: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS

Preface xiii

Codes for Download xvii

1 Modeling and Simulation 1

11 Numerical Approximation 112 C++ for Numerical Modeling 313 Mathematical Modeling 414 Simulation and Its Visualization 615 Numerical Methods 716 Numerical Applications 7

2 Fundamental Tools for Mathematical Computing 13

21 C++ for High-Performance Computing 1322 Dynamic Memory Allocation 1423 Matrix Reduction Problems 2224 Matrix Algebra 3525 Algebra of Complex Numbers 4326 Number Sorting 5127 Summary 54

Programming Challenges 55

3 Numerical Interface Designs 56

31 Microsoft Foundation Classes 5632 Graphics Device Interface 5733 Writing a Basic Windows Program 6034 Displaying Text and Graphics 6835 Events and Methods 6936 Standard Control Resources 7137 Menu and File IO 7838 Keyboard Control 8739 MFC Compatibility with Net 92310 Summary 95

ix

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 12: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

x CONTENTS

4 Curve Visualization 96

41 Tools for Visualization 9642 MyParser 9643 Drawing Curves 10644 Generating Curves Using MyParser 11545 Summary 126

Programming Challenges 126

5 Systems of Linear Equations 127

51 Introduction 12752 Existence of Solutions 12853 Gaussian Elimination Techniques 13154 LU Factorization Methods 14255 Iterative Techniques 16156 Visualizing the Solution Code5 17257 Summary 189

Numerical Exercises 190Programming Challenges 192

6 Nonlinear Equations 193

61 Introduction 19362 Existence of Solutions 19463 Bisection Method 19564 False Position Method 19865 NewtonndashRaphson Method 20166 Secant Method 20367 Fixed-Point Iteration Method 20668 Visual Solution Code6 20869 Summary 225

Numerical Exercises 225Programming Challenges 226

7 Interpolation and Approximation 227

71 Curve Fitting 22772 Lagrange Interpolation 22873 Newton Interpolations 23174 Cubic Spline 23975 Least-Squares Approximation 24476 Visual Solution Code7 24977 Summary 264

Numerical Exercises 265Programming Challenges 265

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 13: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

CONTENTS xi

8 Differentiation and Integration 267

81 Introduction 26782 Numerical Differentiation 26883 Numerical Integration 27184 Visual Solution Code8 27985 Summary 286

Numerical Exercises 286Programming Challenges 287

9 Eigenvalues and Eigenvectors 288

91 Eigenvalues and Their Significance 28892 Exact Solution and Its Existence 28993 Power Method 29194 Shifted Power Method 29295 QR Method 29496 Visual Solution Code9 30297 Summary 322

Numerical Exercises 322Programming Challenges 323

10 Ordinary Differential Equations 324

101 Introduction 324102 Initial-Value Problem for First-Order ODE 325103 Taylor Series Method 327104 RungendashKutta of Order 2 Method 330105 RungendashKutta of Order 4 Method 333106 Predictor-Corrector Multistep Method 335107 System of First-Order ODEs 338108 Second-Order ODE 341109 Initial-Value Problem for Second-Order ODE 3421010 Finite-Difference Method for Second-Order ODE 3451011 Differentiated Boundary Conditions 3511012 Visual Solution Code10 3581013 Summary 378

Numerical Exercises 378Programming Challenges 380

11 Partial Differential Equations 381

111 Introduction 381112 Poisson Equation 385113 Laplace Equation 394114 Heat Equation 397

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 14: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xii CONTENTS

115 Wave Equation 406116 Visual Solution Code11 411117 Summary 437

Numerical Exercises 437Programming Exercises 438

Index 441

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 15: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE

Computing for Numerical Methods Using Visual C++ has been written to promotethe use of Visual C++ in scientific computing C++ is a beautiful language thathas contributed to shaping the modern world today The language has contributedto many device drivers in electronic equipment as a tool in the development ofmany computer software programs and as a tool for both research and teachingTherefore its involvement in providing the solution for numerical methods is verymuch expected

Today research has no boundary A problem for study in a topic in research mayinvolve people from several disciplines A typical problem in engineering for study-ing the effect of chemical spills in a lake may involve engineers chemists biologistsmedical doctors mathematicians economists urban planners and politicians A com-prehensive solution that satisfies all parties can only be produced if people from thesedisciplines cooperate rather than having them acting as rivals

Numerical computing is an important area of research in science and engineeringThe topic is widely implemented in the modeling of a problem and its simulationAdditional work involves visualization which makes the problem and its solutionacceptable to the general audience In the early days of computing in the 1960sand 1970s the solutions to problems were mostly presented as text and numbersA programming language like FORTRAN was the dominant tool and there wereno friendly interfaces to present the solutions Things have improved much sincethen as new advancements in hardware and software produce friendly tools basedon Microsoft Windows Numerical computing benefits much from Windows as theresults from computation can now be visualized as graphs numbers moving imagesas well as text

We select numerical methods as the main title in the book as the concepts inthis topic serve as the fundamentals in science and engineering The importance ofnumerical methods has been proven as nearly all problems involving mathematicalmodeling and simulations in science and engineering have their roots in numericalmethods All numeric-intensive applications involving arrays and vectors have theirconcepts defined in numerical methods Numerical methods discusses vital and crit-ical techniques in implementing algorithms for providing fast reliable and stablesolutions to these problems

Numerical analysis is a branch of mathematics that studies the numerical solutionsto problems involving nonlinear equations systems of linear equations interpolationand approximation for curve fittings differentiation integrals ordinary differentialequations and partial differential equations Numerical solutions are needed for these

xiii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 16: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xiv PREFACE

problems as an alternative to the exact solutions which may be difficult to obtainThe exact solutions for these problems are studied using analytical methods basedon mathematical techniques and theorems The numerical solutions produced maynot be exact but they are good enough for acceptance as they only differ by a smallmargin

The scope for numerical analysis is very broad The study involves the analyticalderivation of the methods or techniques using mathematical principles and rulesThe study also involves a detailed analysis of the errors between the approximatedsolutions and the exact solutions so as to provide faster convergence as well as moreaccurate solutions

Numerical methods is different from numerical analysis Numerical methods isa branch of numerical analysis that specially deals with the implementation of themethods for solving the problems The details about the derivation of algorithmsand techniques for solving the problems and the analysis of errors are not in themain agenda of numerical methods The main objective in numerical methods isapplying the given methods for solving the problems It is the implementation ofthe numerical methods that attracts interest from the practitioners who comprise thebiggest consumer market Engineers scientists and technologists belong to this groupof people who view numerical methods as an important tool for solving their problemsOn the other hand numerical analysis is mostly confined to die-hard mathematicianswho love further challenges in developing new numerical techniques for solving theproblems

There are several objectives in developing Computing for Numerical MethodsUsing Visual C++ First no books on the market today discuss the visual solutions toproblems in numerical methods using C++ There are similar books using softwarepackages such as Matlab Maple and Mathematica These software packages arenot really primitive programming languages They have been developed to hide theprogramming details and to implement the solution as a black box In other wordssoftware packages do not really teach the mathematical concepts and principles insolving a problem For example the inverse of a matrix can be computed using asingle line of command in these packages The user only needs to know the formatand syntax of the command in order to produce the desired solution It is not importantfor the user to know the underlying concepts in solving the problem Computing forNumerical Methods Using Visual C++ is one effort to integrate C++ with the visualsolution to problems using numerical methods

A student cannot be too reliant on software packages There are cases wheresoftware packages fail to provide a solution because of the lack of special routinesFor example a software package may only support a maximum of five levels of therectangular grids in a boundary-value problem involving partial differential equationsTo produce 10 levels the user will have to use C++ as the language because it is moreflexible Flexibility and versatility are two features in C++ that cannot be matched byany software package

Our second objective is to promote C++ as a language for numerical computingC++ has all the necessary ingredients for numerical computing because of its flexiblelanguage format its object-oriented methodology and its support for high numerical

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 17: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

PREFACE xv

precisions However in the past the popularity of C++ has suffered from the emer-gence of several new languages Among them are Java Python and C These newlanguages have been developed with the main objective to handle Web and networkprogramming requirements Other than that C++ is still dominant and practical for im-plementation Because of this reason C++ is still popular in schools and universitiesmostly for teaching and research purposes C++ is also used widely in the manufac-turing sectors such as in the design of device drivers for electronic components

Our third objective is to make numerical problems friendly and approachableThis goal is important as the general public perception about mathematics is that itis tough unfriendly boring and not applicable in daily life A mathematician shouldnot be placed in the basement floor of a building under the feeling that he is notimportant for people to meet A mathematician can become a role model if he canexert his usefulness in a friendly and acceptable way which can be done by makingmathematics interesting and approachable through a series of friendly interfaces Aweak or average student can become motivated with mathematics if the right toolsfor understanding mathematics are provided

A visual approach based on Windows in Computing for Numerical Methods UsingVisual C++ is our step in achieving this objective The book teaches the reader onthe friendly interfaces in tackling problems in numerical methods The interfacesinclude buttons dialog boxes menus and mouse clicks The book also provides avery useful tool called MyParser which can be used to develop various friendlynumerical applications on Windows MyParser is an equation parser that reads anequation input by the user in the form of a string processes the string and producesits solution In promoting its use we hide all technical details in the development ofthe parser so that the reader can concentrate on producing the solution to the problemBut the best part is MyParser is free for distribution for those who are interested

Our last objective is to maintain links with the Microsoft family of products throughthe Net platform Microsoft is unarguably the driver in providing visual solutionsbased on Windows and the Net platform provides a common multilanguage programdevelopment for applications on Windows As Visual C++ is one product supportedin the Net platform there is a guarantee of continued support from Microsoft for itsusers A Visual C++ follower can also enjoy the benefit of integrating her product withother products within the Net platform with very minimum effort This flexibilityis important as migrating from one system to a different system by bringing alongdata and programs can be a very expensive time-consuming and resource-dependentaffair

In providing the solutions this book does not provide detailed coverage of eachtopic in numerical methods There are already many books on the market that docover these topics and we do not wish to compete against them Instead we focuson the development stages of each topic from the practical point of view usingVisual C++ as the tool Knowing how to write the visual interfaces for the numericalproblems will definitely contribute to guiding the reader toward the more ambitiousnumerical modeling and simulation projects This objective is the main benefit thatcan be expected from the book The reader can take advantage of the supplied codesto create several new projects for high-performance computing

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 18: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 836 Char Count= 0

xvi PREFACE

The book is accompanied by source codes that can be downloaded from the givenWiley website This website will be maintained and updated by the authors from timeto time The codes have been designed to be as compact as possible to make themeasy to understand They are based on the Microsoft Foundation Classes library forproviding the required user-friendliness tools In designing the codes we opted for theunguided (or non-wizard) approach in order to show the detailed steps for producingthe output This is necessary as the guided (or wizard) approach does not teach somekey steps and it causes the program to be extremely long in size However the readershould be able to convert each unguided program code to the guided code if the needarises

In preparing the manuscript for Computing for Numerical Methods Using VisualC++ the authors would like to thank several people who have been directly orindirectly involved The authors would like to thank Tan Sri Prof Dr Ir Mohd ZulkifliTan Sri Mohd Ghazali Vice Chancellor of Universiti Teknologi Malaysia for hisforward vision in leading the university towards becoming a world-class university by2010 Special thanks also to Professor Dr Alias Mohd Yusof and Professor Dr MdNor Musa from Universiti Teknologi Malaysia and Professor Stephan Olariu fromOld Dominion University for their support and encouragement

Shaharuddin SallehAlbert Y ZomayaSakhinah Abu BakarApril 2007

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 19: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

CODES FOR DOWNLOAD

All codes discussed in this book can be downloaded from the following URL

ftpftpwileycompublicsci tech medcomputing numerical

The files in the URL are organized into a directory called SALLCode The codesfor the program are located in the folders bearing the chapter numbers for exampleCode4 is the project for Chapter 4 The files included are the executable (exe) header(h) C++ (cpp) and the parser object file MyParserobj

The ftp site will constantly be maintained and updated Any questions commentsand suggestions should be addressed to the first author at ssutmmy

The system requirements for the codes are as follows

Intel Pentium-based Personal computer with 256 MB RM and above

Microsoft Windows 1998 and above

Microsoft Visual C++ version 6 and above

All code files have been tested using Microsoft Visual C++Net version 2003 Thesame files should be compatible with Visual C++ version 6 and below and withMicrosoft Visual C++Net version 2005 and above

xvii

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 20: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-FM JWDD068-Salleh October 11 2007 1053 Char Count= 0

xviii

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 21: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

CHAPTER 1

Modeling and Simulation

11 Numerical Approximations12 C++ for Numerical Modeling13 Mathematical Modeling14 Simulation and Its Visualization15 Numerical Methods16 Numerical Applications

References

11 NUMERICAL APPROXIMATIONS

Numerical methods is an area of study in mathematics that discusses the solutionsto various mathematical problems involving differential equations curve fittingsintegrals eigenvalues and root findings through approximations rather than exactsolutions This discussion is necessary because the exact solutions to these problemsare difficult to obtain through the analytical approach For example it may be wiseto evaluate

int 5

minus1x sin x dx

as the exact solution can be obtained through a well-known technique in calculuscalled integration by parts However it is not possible to apply the same method orany other analytical method to solve

int 2

0

eminusx

3 sin x + x2dx

The given equation in the above integral is difficult to solve as it is not subjectto the exact methods discussed in ordinary calculus Therefore a numerical methodis needed to produce a reasonably good approximated solution A good approxi-mated solution whose value may differ from the exact solution by some fractions isdefinitely better than nothing

1

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 22: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

2 MODELING AND SIMULATION

Numerical methods are also needed in cases where the mathematical function fora given problem is not given In many practical situations the governing equationsfor a given problem cannot be determined Instead an engineer may have a set of ndata (xi yi ) collected from the site to analyze in order to produce a working modelIn this case a numerical method is applied to fit a curve that corresponds to this setof data for modeling the scenario

Solutions to numerical problems can be obtained on the computer in two waysusing a ready-made software program or programming using a primitive languageA ready-made software program is a commercial package that has been designedto solve specific problems without the hassle of going through programming Thesoftware provides quick solutions to the problems with just a few commands Thesolution to a problem such as finding the inverse of a matrix is obtained by typingjust one or two lines of command Matlab Maple and Mathematica are some of themost common examples of ready-made software programs that are tailored to solvenumerical problems

The easy approach using ready-made software programs has its drawback Thesoftware behaves like a black box where the user does not need to know the details ofthe method for solving the problem The underlying concepts in solving the problemare hidden in the software and this approach does not really test the mathematicalskill of the user Very often the user does not understand how the method works asall he or she gets is the generated solution In addition the user may face difficultyin trying to figure out why a particular solution fails because of problems such assingularity in the domain

A ready-made software program also does not provide the flexibility of customizingthe solution according to the userrsquos requirement The user may need special featuresto visualize the solution but these features may not be supported in the software Inaddition a ready-made software program generates files that are relatively large insize The large size is the result of the large number of program modules from itslibrary that are stored in order to run the program

The real challenge in solving a numerical problem is through the native languageprogramming It is through programming that a person will understand the wholemethod comprehensively The developer will need to start from scratch and willneed to understand all the fundamental concepts for solving a given problem beforethe solution to the problem can be developed The whole process in providing thesolution may take a long time but a successful solution indicates the programmerfully understands the whole process

It is also important for us to accept both approaches Ready-made software pro-grams are needed in cases where the program needs to be delivered fast A ready-madesoftware program can produce the desired solution within a short period of time ifall required routines are available in the software which is done by following theright commands and procedures in handling the software In cases where some re-quired actions are not supported in the software it may be necessary to integrate thesolution with a programming language such as C++ Conversely if one starts from aprogramming language it may be necessary call a ready-made software program tohandle some difficult tasks For example C++ may be used to draw up the numerical

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 23: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

C++ FOR NUMERICAL MODELING 3

solution to the heat distribution problem To see the solution in the form of surfacegraph it may be wise to call a few routines from Matlab as the same feature in C++will require a long time to develop

12 C++ FOR NUMERICAL MODELING

C++ is a language that has its origin from C developed in early 1980s C++ retainsall the procedural structure of C but adds the object-oriented features in order tomeet the new requirements in computing Both C and C++ are heavily structuredhigh-level languages that produce small executable files The two languages are alsosuitable for producing low-level routines that run the device drivers of many electroniccomponents

C++ is popular because of its general-purpose features to support a wide varietyof applications such as data processing numerical scientific and engineering C++is available in all computing platforms including Windows UNIX Macintosh andoperating systems for mainframe and minicomputers C++ is a revolutionary languagethat has a very strong following from students practitioners researchers and softwaredevelopers all over the world The language is taught in most universities and collegesin the world as a one- or two-semester subject to support numerical and general-purpose applications

C++ is a language that strongly supports object-oriented programming Object-oriented programming is a programming approach based on objects An object is aninstance of a class A class is a set of entities that share the same parent As it stoodC++ is one of the most popular object-oriented programming languages in the worldThe main reason for its popularity is because it is a high-level language but at thesame time it runs as powerful as the assembly language But the real strength of C++lies in its takeover from C to move to the era of object-oriented programming in thelate 1980s This conquest provides C++ with the powerful features of the procedural Cand an added flavor for object-oriented programming

The original product from Microsoft consists of the C compiler that runs under theMicrosoft DOS (disk operating system) and it has been designed to compete againstTurbo C which was produced by the Borland Corp In 1988 C++ was added to Cand the compiler was renamed Microsoft C++ In early 1989 Microsoft launchedthe Microsoft Windows operating system which includes the Windows API (appli-cation programming interface) This interface is based on 16 bits and it supports theprocedural mode of programming using C

Improvements were made over the following years that include the WindowsSoftware Development Kit (SDK) This development takes advantage of the APIfor the graphical user interface (GUI) applications with the release of the Microsoft Ccompiler As this language is procedural the demands in the applications requirean upgrade to the object-oriented language design approach and this contributesto the release of the Microsoft C++ compiler With the appearance of the 32-bitWindows API (or Win32 API) in the early 1990s C++ was reshaped to tackle theextensive demands on Windows programming and this brings about the release ofthe Microsoft Foundation Classes (MFC) library The library is based on C++ and it

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 24: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

4 MODELING AND SIMULATION

has been tailored with the object-oriented methodology for supporting the applicationarchitecture and implementation

The main reason why Visual C++Net is needed in numerical methods is its pow-erful simulation and visualization tools The Net platform refers to a huge collectionof library functions and objects for creating full-featured applications on both thedesktop and the enterprise Web The classes and objects provide support for friendlyuser interface functions like multiple windows menus dialog boxes message boxesbuttons scroll bars and labels Besides the platform also includes several tedioustask-handling jobs like file management error handling and multiple threading Thisplatform also supports advanced frameworks and environments such as the PassportWindows XP and the Tablet PC The strength of the Net platform is obvious in pro-viding the Internet and Web enterprise solutions Web services include informationsharing e-commerce HTTP XML and SOAP XML or Extensible Markup Lan-guage is a platform-independent approach for creating markup languages needed ina Web application

A new approach in Visual C++Net is the Managed Extension which performsautomatic garbage collection for optimizing the code Garbage collection involvesthe removal of memory and resources unused any more in the application whichis often neglected by the programmer The managed extension is a more structuredway in programming and it is now the default in Visual C++Net Central to the Netplatform is the Visual Studio integrated development environment (IDE) It is in thisplatform that applications are built from a choice of several powerful programminglanguages that include Visual Basic Visual C++ Visual C and Visual J++

In addition IDE also provides the integration of these languages in tackling a par-ticular problem under the Net banner Visual C++Net is one of the high-performancecompilers that makes up the NET platform This highly popular language has its rootin C and was improved to include the object-oriented elements now with the Netextension it is capable of creating solutions for the Web enterprise requirements Arelatively new language called Visual C in the Net family was developed by takingthe best features from Visual Basic visual tools with the programming power of VisualC++

In addition to its single-machine prowess Visual C++Net presents a powerfulapproach for building applications that interact with databases through ADONETThis product evolves from the earlier ActiveX Data Objects (ADO) technology andit encompasses XML and other tools for accessing and manipulating databases forseveral large-scale applications This feature makes possible Visual C++Net as anideal tool for several Web-based database applications

13 MATHEMATICAL MODELING

Many problems arise in science and engineering that have their roots in mathematicsProblems of this nature are best described through mathematical models that providethe fundamental concepts needed in solving the problem A successful mathematicalmodeling always leads to a successful implementation of the given project

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 25: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

MATHEMATICAL MODELING 5

A mathematical model is an abstract model that uses mathematical language todescribe the behavior of a system It is an attempt to find the analytical solutions forenabling the prediction of the behavior of the system from a set of parameters andtheir initial and boundary conditions in a given problem A mathematical model iscomposed of variables and operators to represent a given problem Variables are theabstractions of the quantities of interest in the described systems whereas operatorsare the mechanisms that act on these variables An operator can be expressed in theform of an algebraic operator a function a differential operator and so on

One good example of mathematical modeling is in the heat distribution problemin a two-dimensional plane which is modeled as a Laplace function given by

nabla2u = part2u

partx2+ part2u

party2

In the above equation nabla2 = part2

partx2 + part2

party2 is an operator that acts on the heat quan-tity u whose independent variables are x and y The model describes the analyt-ical solution of heat distribution in a given domain subject to certain initial andboundary conditions We will discuss the numerical solution to this problem later inChapter 11

A mathematical model is often represented as variables in terms of objectivefunctions and constraints If the objective functions and constraints in the problemare represented entirely by linear equations then the model is regarded as a linearmodel If one or more of the objective functions or constraints are represented with anonlinear equation then the model is known as a nonlinear model Most problems inscience and engineering today are modeled as nonlinear

A deterministic model is one where every set of variable states is uniquely deter-mined by parameters in the model and by sets of previous states of these variablesFor example the path defined by a delivery truck for distributing petrol in a city isdefined as a deterministic model in the form of a graph In this case the petrol stationscan be modeled as the nodes of the graph whereas the path is defined as the edgesbetween the nodes Conversely in a probabilistic or stochastic model randomnessis present and the variable states are not described by unique values Very often themodel is represented in the form of probability distribution functions

A mathematical model can be classified as static or dynamic A static model doesnot account for the element of time whereas a dynamic model does Dynamic mod-els typically are represented with difference equations or differential equations Amodel is said to be homogeneous if it is in a consistent state throughout the entire sys-tem If the state varies according to certain controlling mechanism then the model isheterogeneous If the model is homogeneous then the parameters are lumped or con-fined to a central depository A heterogeneous model has its parameters distributedDistributed parameters are typically represented with ordinary or partial differentialequations

Mathematical modeling problems are also classified into black-box or white-boxmodels according to how much a priori information is available in the system Ablack-box model is a system of which no a priori information is available A white-box

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 26: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

6 MODELING AND SIMULATION

model also called glass box or clear box is a system where all necessary informationis available

14 SIMULATION AND ITS VISUALIZATION

Mathematical modeling is often followed by a series of numerical simulations tosupport and verify its validity and correctness Simulation is an imitation of some realthing state of affairs or process representing certain key characteristics or behaviorsof a selected physical or abstract system It is also the objective of a simulation to op-timize the results by controlling the variables that make up the problem By changingthe variables during simulation predictions may be made about the behavior of thesystem Simulation is necessary to save time cost human capital and other resourcesGood results from a simulation contribute in some critical decision-making process

Simulation can be implemented using three approaches microscopic macro-scopic and mesoscopic In the microscopic simulation the detail physical and per-formance characteristics such as the properties of the elements that make up theproblem are considered The simulation involves some tiny properties of the individ-uals or elements that make up the pieces The results from a microscopic simulationare always reliable and accurate However this approach could be very costly andtime consuming as data from the individuals or elements are not easy to obtain

An easier approach is the macroscopic simulation that considers the deterministicfactors of the whole population rather than all the individuals or elements In this ap-proach factors such as the governing mathematical equations and their macro data areconsidered The steps in this approach may skip the detail components and thereforethe results may not be as accurate as the one produced in the microscopic approachHowever this approach saves time and is not as costly as the microscopic approach

A more realistic and practical approach is the mesoscopic simulation that combinesthe good parts of the microscopic and macroscopic approaches to produce a moreversatile model In this approach some deterministic properties of the elements inthe system are integrated with the detail information to produce a workable model

A good simulation has several visualization features that accurately describe theelements in a system Visualization is a form of graphical or textual presentationthat easily describes the solution to a particular problem An effective visualizationincludes components such as text graphics diagrams images animation and soundin order to describe the system

In most cases numerical simulations are carried out effectively in a computer Acomputer can accurately describe the functionality and behavior of the elements thatmake up the system Todayrsquos computers are fast and have all the required resources toperform most college-level numerical simulations Supercomputers also exist that aremultiprocessor systems capable of processing numeric-intensive applications witha whopping gigaflop speed Several parallel and distributed computer systems arealso available in processing these numeric-intensive applications Computers are alsogrouped into clusters to work cooperatively in grid computing networks that spanacross many countries in the globe

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 27: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL METHODS 7

15 NUMERICAL METHODS

Numerical methods is a branch of mathematics that consists of seven core areas asfollows

Nonlinear equations System of linear equations Interpolation and approximation Differentiation and integration Eigenvalues and eigenvectors Ordinary differential equations Partial differential equations

The basic problem in a nonlinear equation is in finding the zeros of a given functionThe problem translates into finding the roots of the equation or the points along thex-axis where the function crosses The roots may exist as real numbers In cases wherethe real roots do not exist their corresponding imaginary roots may become a topicof study

Linear equations are often encountered in various science and engineering prob-lems The problem arises frequently as the original problem reduces to linear equationsat some stage in the solution For example mesh modeling using the finite-elementmethod in a fluid dynamics results in a system of hundreds of linear equations Asthe size of the matrix in this problem is large the solution needs to be tackled usinga fast numerical method

Interpolation and approximation are curve fitting problems that contribute in thingslike designing the surface of an aircraft The techniques are also applied in otherproblems such as forecasting pattern matching and routing

Differentiation and integration are fundamental topics that arise in many problemsGood approximations are needed to these two topics as their exact values may not beeasy to obtain

Problems involving ordinary differential equations arise in modeling and simu-lation Exact solutions are difficult to obtain as the models are subject to variationbecause of the presence of constraints and nonlinear factors Therefore numericalmethods are needed in their successful implementation

Modeling and simulation involving partial differential equations commonly usenumerical techniques as their fundamental elements Numerical methods contributeto provide the desired solutions in most cases as the exact solutions are not practicalfor implementation on the computer

16 NUMERICAL APPLICATIONS

Most problems in science and engineering are inherently nonlinear in nature Thisnonlinearity is because the problems are dependent on variables and parametersthat are nonlinear and are subject to many constraints Many problems are also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 28: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

8 MODELING AND SIMULATION

dynamic and nondeterministic with no a priori information Because of their naturethe solution to these problems will not be a straightforward task

In most cases the normal approach for solving nonlinear problems in science andengineering is to start from the fundamental concepts that are based on mathematicsA mathematical model that describes the system needs to be developed to representthe problem Data from the problem are collected as an input Numerical simulationbased on the theoretical model is then performed on the data During the simulationthe results obtained are periodically compared and matched with some real values inorder to verify the correctness of the simulation

We discuss some common modeling and simulation work that makes use of nu-merical methods

Bacteria Population Growth

Population growth of bacteria in a geographical region over a period of time has beensuccessfully modeled using a differential equation given as

dx

dt= kx

where x(t) represents the size of the population at time t and k is a constant In abroader scope x(t) in the above equation may represent the number of bacteria in asample container

The numerical solution to this model consists of solving an initial value probleminvolving the first-order ordinary differential equation A suitable solution is providedin the form of the RungendashKutta method of order 4 as will be discussed in Chapter 10However the solution is not solely provided by this method alone Constraints suchas temperature and the acidity of the fluid in the container may affect the validity ofthe mathematical model The bacteria may grow faster when the temperature is highand when the acidity of the fluid is low Therefore a numerical simulation is neededto integrate the mathematical model with other variables and parameters Severalparameters can be included in the simulation in order to produce the correct modelfor this problem

Computational Fluid Dynamics

Computational fluid dynamics (CFD) is an area of research that deals with the dynamicbehavior and movement of fluid under certain physical conditions Numerical methodsand algorithms are used extensively to solve and analyze problems involving fluid flowFor example the interaction between particles in fluids and gases are studied throughsimulation on the computer Millions of calculations are performed in this simulationas the original problem reduces to numerical problems such as matrix multiplicationsmatrix inverse system of linear equations and the computation of eigenvalues

One area of study in computational fluid dynamics is the blood flow modeling instenosed artery of the human body The study contributes in predicting the occurrenceof cardiovascular diseases such as heart attack and stroke CFD simulations have also

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 29: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 9

been carried out in the aerospace and automotive industries for evaluating the air flowaround moving aircrafts and cars

The fundamental tool in CFD simulation is the NavierndashStokes equation whichdescribes a single-phase fluid flow Also a set of ordinary or partial differentialequations with their initial and boundary conditions is given The solution to theseproblems makes use of the finite-difference method or finite-element methods whichreduces the problem to several systems of linear equations

Finite-Element Modeling

The finite-element method is a numerical technique for evaluating things like stressesand displacements in mechanical objects and systems The finite-element methodhas been successfully applied for modeling problems involving heat transfer fluiddynamics electromagnetism and solid state diffusion At the National Aeronauticsand Space Administration (NASA) the finite-element method has been applied inmodeling the turbulence that occurs during the aircraft flight

The finite-element method requires the domain in the problem to be divided intoseveral elements in the form of line segments triangles rectangular meshes volumeand so on The method provides flexibility where the elements need not be of equalin terms of dimension and size Solutions are obtained from these elements and theyare grouped to produce the overall solution

The fundamentals of finite-element methods rest heavily on numerical methodsThey include curve and surface fittings using interpolation or approximation tech-niques systems of linear equation and ordinary and partial differential equations

Printed-Circuit Board Design

Massive simulations are carried out to produce optimal designs for printed-circuitboards (PCBs) A PCB forms the main circuitry of all electronic devices A typicalPCB can accommodate thousands or millions of microelectronic components suchas pins vias transistors processors and memory chips In addition the PCB hasmassive wirings that connect these components

As the space on a PCB is limited the designer must optimize the placement of com-ponents and its routing (wiring) so that the board is capable of accommodating as manycomponents as possible within the limited space area We can imagine a PCB functionlike a city where the buildings and streets need to be designed properly so that the citywill not be too congested with problems such as improper housing and traffic jams

One technique commonly applied for routing in the PCB design is single-rowrouting The problem is about designing non-crossing tracks between pairs of pinsthat are arranged in a single row in such a way that the tracks do not cross Single-rowrouting has been known to be NP-complete with many interacting degrees of freedomFigure 11 shows an optimal output from single-row routing involving 42 pins Theproven methods for solving this problem involve computer simulations using graphtheory simulated annealing and genetic algorithm A technique from the authorscalled ESSR in 2003 has been successful in producing optimal results for the problem

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 30: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

10 MODELING AND SIMULATION

1 2 3 4 5 6 78

9 10 11 12 13 14 1516 17

18 19 20 21

22 23

24 25 26 27 28 29 30

31 32 33

34 35 36

37 38 39 40 41

42

FIGURE 11 Optimal single-row routing involving 42 pins

Wireless Sensor Networks

A sensor network is a deployment of massive numbers of small inexpensive self-powered devices called motes that can sense compute and communicate with otherdevices for the purpose of gathering local information to make global decisions abouta physical environment The object of the network is to detect certain items over ageographical region such as the presence of harmful chemical bacteria electromag-netic field and temperature distribution In one scenario thousands of tiny sensormotes are distributed from an aircraft over a region Sensor network research origi-nates from a DARPA project called Smartdust in late 1990s [2] The research attractsinterest from many disciplines because of several developments in the micro-electromechanical system (MEMS) technology that produces many cheap and small sensormotes

Locating sensor motes at their geographical location has become one major issue informing the network because many constraints need to be considered in the problemEach node in a sensor network has a short lifetime based on a battery To save energythe node sleeps most of the time and becomes awake only occasionally Thereforethe process of training the nodes in order to locate their correct location is a highlynonlinear problem with many constraints

A coarse-grain model developed in Ref 3 proposed a method called asynchronoustraining for locating the nodes in a sensor network Figure 12 shows a model con-sisting of three networks for training the sensor nodes Each network is indicated byconcentric circles that originate from a center called sink The sink is representedby a device called the aggregating and forwarding node (AFN) which has a pow-erful transmitter and receiver for reaching all nodes in its network In this modelAFN is responsible for training the nodes storing and retrieving data from the sensornodes and performing all the necessary computations on the data received to producethe desired results The model makes use of the dynamic coordinate system based oncoronas and wedges to locate the nodes through a series of transmission from the sink

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 31: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

NUMERICAL APPLICATIONS 11

AB

C

FIGURE 12 The asynchronous model for training the sensor nodes

This theoretical model is not implemented yet in the real world but it has a strongpotential for applications Modeling and simulation on the computer definitely helpsin convincing the lawmakers to realize the benefit from this research

Flood Control Modeling

The city of Kuala Lumpur Malaysia recently completed the construction of a tunnelthat has dual purposes as a road during the normal time and as a waterway tunnelwhenever flood hits the city The project is called SMART or the Stormwater Man-agement and Road Tunnel This project is the first of its kind in the world and itcontributes in controlling flash flood which frequently disrupts communication inthe city

SMART is an underground tunnel about 10 km long that was constructed usingtunnel-boring machines The tunnel connects two rivers one in the middle of the cityand the other away from the city During the normal days the tunnel serves as twotrunk roads going into and out of the city and this contributes in reducing traffic jamin some parts of the city During heavy rain parts of the city get flooded as wateroverflows from the main river in the city The tunnel is immediately closed to trafficand it is converted into a waterway to divert the water from the flood area into thesecond river This has the immediate effect of reducing the occurrence of flood inthese areas When the flood ends the tunnel is cleaned and it is open for traffic again

A lot of work involving modeling and simulation has been carried out before theproject gets started As the project is costly modeling and simulation using computer

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 32: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOSPI

JWDD068-Salleh-CH-01 JWDD068-Salleh September 15 2007 610 Char Count= 0

12 MODELING AND SIMULATION

help in providing all the necessary input before a decision on the viability of theproject is made The heavy costs involved are well justified here

REFERENCES

1 S Salleh B Sanugi H Jamaluddin S Olariu and A Y Zomaya Enhanced simulatedannealing technique for the single-row routing problem Journal of Supercomputing 21(3)2002 285ndash302

2 S Olariu and Q Xu A simple self-organization protocol for massively deployed sensornetworks Computer Communications 28 2005 1505ndash1516

3 Q Xu R Ishak S Olariu and S Salleh On asynchronous training in sensor networksJournal of Mobile Multimedia 3(1) 2007

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 33: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

CHAPTER 2

Fundamental Tools for MathematicalComputing

21 C++ for High-Performance Computing22 Dynamic Memory Allocation23 Matrix Elimination Problems24 Matrix Algebra25 Algebra of Complex Numbers26 Number Sorting27 Summary

Programming Challenges

21 C++ FOR HIGH-PERFORMANCE COMPUTING

Engineering problems involving large arrays with high-resolution graphics for vi-sualization are some of the typical applications in high-performance computing Inmost cases high-performance computing is about intensive numerical computingover a large problem that requires good accuracy and high-precision results High-performance computing is a challenging area requiring advanced hardware as well asa nicely crafted software that makes full use of the resources in the hardware

Numerical applications often involve a huge amount of computation that requiresa computer with fast processing power and large memory Fast processing capabilityis necessary for calculating and updating several large size arrays which represent thecomputational elements of the numerical problem Large memory is highly desirableto hold the data which also contributes toward speeding up the calculations

C++ was originally developed in the early 1980s by Bjarne Stroustrup The featuresin C++ have been laid out according to the specifications laid by ANSI (AmericanNational Standards Institute) and ISO (International Standards Organization) Thelanguage is basically an extension of C that was popular in the 1970s replacingPascal and Fortran C is a highly modular and structured high-level language whichalso supports low-level programming The language is procedural in nature and it haswhat it takes to perform numerical computing besides being used for general-purposeapplications such as in systems-level programming string processing and databaseconstruction

13

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 34: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

14 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Primarily C++ extends what is lacking in C namely the object-oriented approachto programming for supporting todayrsquos complex requirements Object-oriented pro-gramming requires a problem to be broken down into objects which can then behandled in a more practical and convenient manner That is the standard put forwardfor meeting todayrsquos challenging requirements The programming language must pro-vide support for exploring the computerrsquos resources in providing an easy and friendly-looking graphical user interface for the given applications The resources are providedby the Windows operating system which includes the mousersquos left and right buttonclicks text and graphics keys from the keyboard dialog boxes and multimedia sup-port In addition C++ is a scalable language that makes possible the extension ofthe language to support several new requirements for meeting todayrsquos requirementsincluding the Internet parallel computing wireless computing and communicationwith other electronic devices

In performing scientific computing several properties for good programming helpin achieving efficient coding They include

Object-oriented methodology Modular and structured program design Dynamic memory allocation for arrays Maximizing the use of local variables Encouraging data passing between functions

In this chapter we discuss several fundamental mathematical tools and their pro-gramming strategies that are commonly deployed in high-performance computingThey include strategies for allocating memory dynamically on arrays passing ar-ray data between functions performing algebraic computation on complex numbersproducing random numbers and performing algebraic operations on matrices As thenames suggest these tools provide a window of opportunity for high performancecomputing that involves vast areas of numeric-intensive applications

22 DYNAMIC MEMORY ALLOCATION

A variable represents a single element that has a single value at a given time An arrayin contrast is a set of variables sharing the same name where each of them is capableof storing one value at a single time An array in C++ can be in the form of one ormore dimensions depending on the program requirement An array is suitable forstoring quantities such as the color pixel values of an image the temperature of thepoints in a rectangular grid the pressure of particles in the air and the values returnedby sensors in an electromagnetic field

An array is considered large if its size is large One typical example of large arraysis one that represents an image A fine and crisp rectangular image of size 1 MBproduced from a digital camera may have pixels formed from hundreds of rows andcolumns Each pixel in the image stores an integer value representing its color in thered green and blue scheme

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 35: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 15

A computer program having many large arrays often produces a tremendousamount of overhead to the computer The speed of execution of this program isvery much affected by the data it carries as each array occupies a substantial amountof memory in the computer However a good strategy for managing the computerrsquosmemory definitely helps in overcoming many outstanding issues regarding large ar-rays One such strategy is dynamic memory allocation where memory is allocated toarrays only when they are active and is deallocated when they are inactive

In scientific computing a set of variables that have the same dimension and lengthis often represented as a single vector Vector is an entity that has magnitude or lengthand direction Opposite to that is a scalar which has magnitude but does not havedirection For example the mass of an object is a scalar whereas its weight subjectto the gravitational pull toward the center of the earth is a vector

Allocation for One-Dimensional Arrays

In programming a vector is represented as a one-dimensional array A vector v ofsize N has N elements defined as follows

v = [v1 v2 middot middot middot vN ]T =

v1

v2

middot middot middotvN

(21)

The above vector is represented as a one-dimensional array as follows

v[1]v[2]v[N]

The magnitude or length of this vector is

|v| =radic

v21 + v2

2 + middot middot middot + v2N =

radicradicradicradic Nsumi=1

v2i (22)

In numerical applications this vector can be declared as real or integer dependingon the data it carries The length of the vector needs to be declared as a floating-point asthe value returned involves decimal points It is important to declare the variable cor-rectly according to the variable data type as wrong declaration could result in data loss

As the size of an array can reach several hundreds thousands or even millionsan optimum mechanism for storing its data is highly desirable For a small size arraystatic memory allocation is a normal way of storing the data An array v with N+1elements is declared statically as

int v[N+1]

This array consists of the elements v[0] v[1] v[N] and memory from thecomputerrsquos RAM has been set aside for this array no matter whether all elements inthe array are active Having only v[1] v[2] and v[3] active out of N = 100 from the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 36: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

16 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

above declaration is clearly a waste as memory has been allocated to all 100 elementsin the array This mode of allocating memory is called static memory allocation

A more practical way of allocating memory is the dynamic memory allocationwhere memory from the computer is allocated to the active elements only in thearray

In dynamic memory allocation the same array whose maximum size is N+1 isdeclared as follows

int vv=new int v[N+1]

Memory for a one-dimensional array is allocated dynamically as a pointer usingthe new directive The allocated size in the declaration serves as the upper limitmemory is allocated according to the actual values assigned to the elements in thearray For example the array v above is allocated with a maximum of N+1 elementsbut memory is assigned to three variables that are active only as follows

int v declaring a pointer to the

integer array

v = new int [N+1] allocating memory of size N+1

v[1]=7 v[2]=-3 v[3]=1 assigning values to the array

delete v destroying the array

Once the array has completed its duty and is no longer needed in the program itcan be destroyed using the delete directive The destruction means the array can nolonger be used and the memory it carries is freed In this way the memory can beused by other modules in the program thus making the system healthier

Dynamic memory allocation for one-dimensional arrays is illustrated using anexample in multiplying two vectors The multiplication or dot product of two vectorsof the same size n u = [u1 u2 middot middot middot uN ]T and v = [v1 v2 middot middot middot vN ]T produces a scalaras follows

u v = u1v1 + u2v2 + middot middot middot + uN vN (23)

This task is implemented in C++ with the result stored in w in the following code

w=0for (i=1ilt=Ni++)

w += u[i]v[i]

The above code has a complexity of O(N ) Code2Acpp shows a full C++ programthat allocates memory dynamically to the one-dimensional arrays u and v and itcomputes the row product between the two arrays

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 37: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 17

Code2Acpp dot-product of two vectorsinclude ltiostreamhgtdefine N 3void main()

int iint u v wu=new int [N+1]u[1]=4 u[2]=-5 u[3]=3cout ltlt endl ltlt Vector u ltlt endlfor (i=1ilt=Ni++)

cout ltlt u[i] ltlt cout ltlt endl

v=new int [N+1]v[1]=-2 v[2]=3 v[3]=7cout ltlt Vector v ltlt endlfor (i=1ilt=Ni++)

cout ltlt v[i] ltlt cout ltlt endl

w=0for (i=1ilt=Ni++)

w += u[i]v[i]cout ltlt the product w=uv is ltlt w ltlt endldelete uv

Allocation for Higher Dimensional Arrays

A two-dimensional array represents a matrix that consists of a set of vectors placedas its columnar elements Basically a two-dimensional array q with M+1 rows andN+1 columns is declared as follows

int q[M+1][N+1]

The above declaration is the static way of allocating memory to the array Thedynamic memory allocation method for the same array is shown as follows

int q declaring a pointer to the

integer array

q = new int [M+1] allocating memory of size

M+1 to the rows

for (int i=0ilt=Mi++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 38: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

18 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

q[i]=new int [N+1] allocating memory of size

N+1 to the columns

for (i=0ilt=Ni++)delete q[i] deallocating the columnar

array memory

delete q destroying the array

In the above method a double pointer to the variable is needed because the arrayis two-dimensional Memory is first allocated to the M+1 row vectors in q Each rowvector has N+1 columns and therefore another chunk of memory is allocated to eachof these vectors Hence a maximum total of [M+1][N+1] amount of memory isallocated to this array

Case Study Matrix Multiplication Problem

We illustrate an important tool in linear algebra involving the multiplication of twomatrices whose memories are allocated dynamically Suppose A = [aik] and B =[bkj ] are two matrices with i = 1 2 M representing the rows of A and j =1 2 N are the columns in B The basic rule allowing two matrices to be multipliedis the number of columns in the first matrix must be equal to the number of rows ofthe second matrix which is k = 1 2 P The product of A and B is then matrixC = [ci j ] or

AB = C

For example if A and B are matrices of size 3 times 4 and 4 times 2 respectively thentheir product is matrix C with the size of 3 times 2 as follows

C =

c11 c12

c21 c22

c31 c32

=

a11 a12 a13 a14

a21 a22 a23 a24

a31 a32 a33 a34

b11 b12

b21 b22

b31 b32

b41 b42

=

a11b11 + a12b21 + a13b31 + a14b41 a11b12 + a12b22 + a13b32 + a14b42

a21b11 + a22b21 + a23b31 + a24b41 a21b12 + a22b22 + a23b32 + a24b42

a31b11 + a32b21 + a33b31 + a34b41 a31b12 + a32b22 + a33b32 + a34b42

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 39: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 19

Obviously each element in the resultant matrix is the sum of four terms that canbe written in a more compact form through their sequence For example c11 has thefollowing entry

c11 = a11b11 + a12b21 + a13b31 + a14b41

which can be rewritten in a simplified form as

c11 =4sum

k=1

a1kbk1

Other elements can be expressed in the same general manner The elements inmatrix C become

C =

c11 c12

c21 c22

c31 c32

=

4sumk=1

a1kbk1

4sumk=1

a1kbk2

4sumk=1

a2kbk1

4sumk=1

a2kbk2

4sumk=1

a3kbk1

4sumk=1

a3kbk2

Looking at the sequential relationship between the elements the following simpli-fied expression represents the overall solution to the matrix multiplication problem

C = [ci j ] =[

Psumk=1

aik bk j

](24)

for i = 1 2 M and j = 1 2 N This compact form is also the algorithmicsolution for the matrix multiplication problem as program codes can easily be designedfrom the solution

We discuss the program design for the above problem Three loops with the iteratorsi j and k are required As k is the common variable representing both the row numberof A and the column number of B it is ideally placed in the innermost loop The middleloop has j as the iterator because it represents the column number of C The outermostloop then represents the row number of C having i as the iterator

The corresponding C++ code for the multiplication problem has the complexityof O(MNP ) and the code segments are

for (i=1 ilt=M i++)for (j=1 jlt=N j++)

c[i][j]=0for (k=1 klt=P k++)

c[i][j] += a[i][k]b[k][j]

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 40: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

20 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Code2Bcpp is a full program for the multiplication problem between the matricesA = [aik] and B = [bkj ] with i = 1 2 M and j = 1 2 N The programdemonstrates the full implementation of the dynamic memory allocation scheme fortwo-dimensional arrays

Code2Bcpp multiplying two matricesinclude ltiostreamhgtdefine M 3define P 4define N 2

void main()

int ijkint abca = new int [M+1]b = new int [P+1]c = new int [M+1]for (i=1ilt=Mi++)

a[i]=new int [P+1]for (j=1jlt=Pj++)

b[j]=new int [N+1]for (k=1klt=Mk++)

c[k]=new int [N+1]

a[1][1]=2 a[1][2]=-3 a[1][3]=1 a[1][4]=5a[2][1]=-1 a[2][2]=4 a[2][3]=-4 a[2][4]=-2a[3][1]=0 a[3][2]=-3 a[3][3]=4 a[3][4]=2cout ltlt Matrix A ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Pj++)cout ltlt a[i][j] ltlt

cout ltlt endl

b[1][1]=4 b[1][2]=-1b[2][1]=3 b[2][2]=2b[3][1]=1 b[3][2]=-1b[4][1]=-2 b[4][2]=4cout ltlt endl ltlt Matrix B ltlt endlfor (i=1 ilt=P i++)

for (j=1 jlt=N j++)

cout ltlt b[i][j] ltlt

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 41: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

DYNAMIC MEMORY ALLOCATION 21

cout ltlt endl

cout ltlt endl ltlt Matrix C (A multiplied by B) ltlt endlfor (i=1 ilt=M i++)

for (j=1jlt=Nj++)

c[i][j]=0for (k=1klt=Pk++)

c[i][j] += a[i][k]b[k][j]cout ltlt c[i][j] ltlt

cout ltlt endl

for (i=1ilt=Mi++)

delete a[i]for (j=1jlt=Pj++)

delete b[j]for (k=1klt=Mk++)

delete c[k]delete abc

The declaration and coding for arrays with higher dimensions in dynamic memoryallocation follow a similar format as those in the one- and two-dimensional arraysFor example a three-dimensional array is declared statically as follows

int r[M+1][N+1][P+1]

The following code shows the dynamic memory allocation method for the same array

int r declaring a pointer

to the integer array

r = new int [M+1] allocation to the

first parameter

for (int i=0ilt=Mi++)

r[i]=new int [N+1] allocation to the

second parameter

for (int j=0jlt=Nj++)r[i][j]=new int [P+1] allocation to the

third parameter

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 42: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

22 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

for (i=0ilt=Ni++)

for (j=0jlt=Pj++)delete r[i][j]

delete r[i]delete r destroying the array

23 MATRIX REDUCTION PROBLEMS

One of the most important operations involving a matrix is its reduction to the uppertriangular form U The technique is called row operations and it has a wide rangeof useful applications for solving problems involving matrices Row operation is anelimination technique for reducing a row in a matrix into its simpler form using oneor more basic algebraic operations from addition subtraction multiplication anddivision A reduction on one element in a row must be followed by all other elementsin that row in order to preserve their values

Vector and Matrix Norms

The norm of a vector is a scalar value for describing the size or length of the vectorwith respect to the dimension n of the vector Norm values of a vector are extensivelyreferred especially in evaluating the errors that arise from an operation involving thevector

Definition 21 The norm-n of a vector of size m v = [v1 v2 middot middot middot vm]T is definedas the nth root of the sum to the power of n of the elements in the vector or

vn = n

radicradicradicradic msumi=1

vni = n

radicxn

1 + xn2 + middot middot middot + xn

m (25)

It follows from Definition 21 that the first norm of v = [v1 v2 middot middot middot vn]T calledthe sum of the vector magnitude norm is the sum of the absolute value of the elementsas follows

v1 =msum

i=1

|vi | = |v1| + |v2| + middot middot middot + |vm | (26)

The second norm of v = [v1 v2 middot middot middot vm]T is called the Euclidean vector normwhich is expressed as follows

v = v2 =radicradicradicradic msum

i=1

v2i =

radicv2

1 + v22 + middot middot middot + v2

m (27)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 43: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 23

Another important norm is the maximum vector magnitude norm or vinfin ofv = [v1 v2 middot middot middot vm]T which is the maximum of the absolute value of the elementsThis norm is defined as follows

vinfin = max1leilem

|vi | = max (|v1| |v2| |vm |) (28)

The rule for determining the maximum matrix norm follows from that of a vectorbecause a matrix of size m times n is made up of m rows of n column vectors each Thisis expressed as follows

Ainfin = max1leilem

(|ai1| + |ai2| + middot middot middot + |ain|)

= max(|a11| + |a12| + middot middot middot + |a1n| |a21| + |a22| + middot middot middot + |a2n| |am1|

+ |am2| + middot middot middot + |amn|)

Example 21 Given v = [minus3 5 2]T and A =[minus4 1 2

3 minus1 5minus2 minus1 1

] find v1 v2

vinfin and Ainfin

Solution The calculation is very straightforward as follows

v1 =3sum

i=1

|vi | = |minus3| + |5| + |2| = 3 + 5 + 2 = 10

v = v2 =radicradicradicradic 3sum

i=1

v2i =

radic(minus3)2 + 52 + 22 =

radic38

vinfin = max1leile3

|vi | = max (|minus3| |5| |2|) = max (3 5 2) = 5

Ainfin = max (|minus4| + |1| + |2| |3| + |minus1| + |5| |minus2| + |minus1| + |1|)= max (7 9 4) = 9

Row Operations

Row operations are the key ingredients for reducing a given matrix into a simplerform that describes the behavior and properties of the matrix

Definition 22 Row operation on row i with respect to row j is an operation involvingaddition and subtraction defined as Ri larr Ri + m R j where m is a constant and Ri

denotes row i In this definition Ri on the left-hand side is a new value obtained fromthe update with Ri on the right-hand side as its old value

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 44: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

24 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

Theorem 21 Row operation in the form Ri larr Ri + m R j on row i with respect torow j does not change the matrix values In this case the matrix with the reducedrow is said to be equivalent to the original matrix

Consider matrix A =[

p q r

s t u

]having two rows and three columns Row

operation on row 2 with respect to row 1 is defined as follows

R2 larr R2 + m R1

The addition operation on R2 produces

B =[

p q r

s + mp t + mq u + mr

]

From Theorem 21 the new matrix obtained B is said to have equal values with theoriginal matrix A Another operation is subtraction

C =[

p q r

s minus mp t minus mq u minus mr

]

which results in matrix C having the same values as A and B We say all three matricesare equivalent or

[p q r

s t u

]sim

[p q r

s + mp t + mq u + mr

]sim

[p q r

s minus mp t minus mq u minus mr

]

A useful reduction is the case in which the constant m is based on a pivot elementA pivot element is an element in a row that serves as the key to the operation on thatrow The normal goal of pivoting an element is to reduce the value of an element tozero In matrix C above the element a21 in A can be reduced to 0 by letting m = s

p In this case a11 = p is the pivot element as this element causes other elements inthe row to change their values as a result from the row operation Row operationR2 larr R2 minus s

p R1 produces

C =p q r

0 t minus s

pq u minus s

pr

Matrix Reduction to Triangular Form

Reduction to zero elements is the basis of a technique for reducing a matrix to itstriangular form A triangular matrix is a square matrix where all elements above orbelow the diagonal elements have zero values The name triangular is reflected in the

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 45: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 25

shape of the matrix with two triangles occupying the elements one with zero elementsof the matrix and another with other numbers (including zeros)

An upper triangular matrix is a triangular matrix denoted as U = [ui j ] whereui j = 0 for j lt i Other elements with j ge i can have any value including zero Forexample a 4 times 4 upper triangular matrix has the following form

U =

0 0 0 0 0 0

(29)

where can be any number including 0Opposite to the upper triangular matrix is the lower triangular matrix L = [li j ]

where li j = 0 for i lt j The following matrix is a general notation for the 4 times 4 lowertriangular matrix L

L =

0 0 0

0 0

0

(210)

A square matrix A is reduced to U or L through a series of row operationsRow operation is a technique for reducing a row in a matrix into a simpler form byperforming an algebraic operation with another row in the matrix Matrix A whenreduced to U or L is said to be equivalent to that matrix

In general a square matrix with N rows requires N minus 1 row operations with respectto rows 1 2 N minus 1 A matrix is usually reduced by eliminating all the rows ofone of its columns to zero with respect to the pivot element in the row The pivotelements of matrix A = [ai j ] are aii which are the diagonal elements of the matrixThe reduction of a square matrix into its upper triangular matrix is best illustratedthrough a simple example

Example 22 Find the upper triangular matrix U of the following matrix

A =

8 2 minus1 2

minus2 1 minus3 minus8

2 minus1 7 minus1

1 minus8 1 minus2

Solution There are four rows in the matrix Reduction of the matrix to its uppertriangular form requires three row operations with respect to rows 1 2 and 3

Operations with Respect to the First Row a11 = 8000 is the pivot element of thefirst row and let m = ai1a11 for the rows i = 2 3 4 All elements in the second

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 46: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

26 FUNDAMENTAL TOOLS FOR MATHEMATICAL COMPUTING

third and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma1 j for columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500 m = a21a11 a2 j larr a2 j minus ma1 j

0000 minus1500 7250 minus1500 m = a31a22 a3 j larr a3 j minus ma1 j

0000 minus8250 1125 minus2250 m = a41a22 a4 j larr a4 j minus ma1 j

Operations with Respect to the Second Row a22 = 1500 is the pivot element ofthe second row and therefore m = ai2a22 for the rows i = 3 4 All elements in thethird and fourth rows are reduced to their corresponding values using the relationshipsai j larr ai j minus ma2 j for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000 m = a32a22

0000 0000 minus16750 minus43500 m = a42a22

Operations with Respect to the Third Row a33 = 4000 is the pivot element of thethird row Therefore m = ai3a33 for the row i = 4 All elements in the fourth roware reduced to their corresponding values using the relationship ai j larr ai j minus ma3 j

for the columns j = 1 2 3 4

8000 2000 minus1000 2000

0000 1500 minus3250 minus7500

0000 0000 4000 minus9000

0000 0000 0000 minus81188 m = a43a33

Row operations on A produce U

U =

8 2 minus1 2

0 1500 minus3250 minus7500

0 0 4000 minus9000

0 0 0 minus81188

The C++ program to reduce A to U is very brief and compact The codeis constructed from the equation ai j larr ai j minus makj where m = aikakk and i =k + 1 k + 2 N k = 1 2 N minus 1 and j = 1 2 N as follows

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl

Page 47: COMPUTING FOR NUMERICAL METHODS USING VISUAL C++ · PDF fileKDO/KEH JWDD068-Salleh-FM JWDD068-Salleh October 17, 2007 21:23 Char Count= 0 COMPUTING FOR NUMERICAL METHODS USING VISUAL

KDOKEH

JWDD068-Salleh-CH-02 JWDD068-Salleh October 5 2007 202 Char Count= 0

MATRIX REDUCTION PROBLEMS 27

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

Code2Ccpp shows the full program for reducing A to U with Code2C in as theinput tile for A

Code2Ccpp reducing a square matrix to its upper triangular forminclude ltfstreamhgtinclude ltiostreamhgtdefine N 4

void main()

int ijkdouble aa=new double [N+1]for (i=0ilt=Ni++)

a[i]=new double [N+1]

ifstream InFile(Code2Cin)for (i=1ilt=Ni++)

for (j=1jlt=Nj++)InFile gtgt a[i][j]

InFileclose()

row operations

double mfor (k=1klt=N-1k++)

for (i=k+1ilt=Ni++)

m=a[i][k]a[k][k]for (j=1jlt=Nj++)

a[i][j] -= ma[k][j]

for (i=1ilt=Ni++)

for (j=1jlt=Nj++)cout ltlt a[i][j] ltlt

cout ltlt endl