JinHo Kwak

Sungpyo Hong

Linear Algebra Second Edition

Springer Science+Business Medi~ LLC

JinHo Kwak Sungpyo Hong Department of Matbematics Pohang University of Science

Department of Mathematics Pohang University of Science

and Technology and Technology Pohang, Kyungbuk 790-784 South Korea

Pohang, Kyungbuk 790-784 SouthKorea

Library of Cougress Cataloging-in-PubHeation Data Kwak, lin Ho, 1948-

Linear algebra I lin Ho Kwak, Sungpyo Hong.-2nd ed. p.cm.

Includes bibliographical references and index. ISBN 978-0-8176-4294-5 ISBN 978-0-8176-8194-4 (eBook) DOI 10.1007/978-0-8176-8194-4

1. Algebras, Linear. I. Hong, Sungpyo, 1948- ß. Title.

QAI84.2.K932004 512' .5-dc22

AMS Subject Classifications: 15-01

ISBN 978-0-8176-4294-5 Printed on acid-free paper.

@2004 Springer Science+Business Media New York Originally published by Birkhlluser Boston in 2004

2004043751 CIP

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher Springer Science+Business Media, LLC, except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrievaI, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to property rights.

987654321 SPIN 10979327

www.birkhasuer-science.com

Preface tothe Second Edition

This second edition is based on many valuable comments and suggestions fromreaders of the first edition. In this edition, the last two chapters are interchangedand also several new sections have been added. The following diagram illustrates thedependencies of the chapters.

Chapter 1Linear Equations and Matrices

Chapter 4Linear Transformations

Chapter 5

Inner Product Spaces

Chapter 8Jordan Canonical Forms

Chapter 2Determinants

Chapter 6Diagonalization

Chapter?Complex Vector Spaces

vi Preface to the Second Edition

The major changes from the first edition are the following.

(1) In Chapter 2, Section 2.5.1 "Miscellaneous examples for determinants" isadded as an application .

(2) In Chapter 4, "A homogeneous coordinate system" is introduced for an appli­cation in computer graphics.

(3) In Chapter 5, Section 5.7 "Relations of fundamental subspaces" and Section 5.8"Orthogonal matrices and isometries" are interchanged. "Least squares solutions,""Polynomial approximations" and "Orthogonal projection matrices" are collectedtogether in Section 5.9-Applications.

(4) Chapter 6 is entitled "Diagonalization" instead of "Eigenvectors and Eigen­values." In Chapters 6 and 8, "Recurrence relations," "Linear difference equations"and "Linear differential equations" are described in more detail as applications ofdiagonalizations and the Jordan canonical forms of matrices .

(5) In Chapter 8, Section 8.5 "The minimal polynomial of a matrix" has beenadded to introduce more easily accessible computational methods for Anand eA , withcomplete solutions of linear difference equations and linear differential equations .

(6) Chapter 8 "Jordan Canonical Forms" and Chapter 9 "Quadratic Forms" areinterchanged for a smooth continuation of the diagonalization problem of matrices.Chapter 9 "Quadratic Forms" is extended to a complex case and includes many newfigures.

(7) The errors and typos found to date in the first edition have been corrected .(8) Problems are refined to supplement the worked-out illustrative examples and

to enable the reader to check his or her understanding of new definitions or theorems.Additional problems are added in the last exercise section of each chapter. Moreanswers, sometimes with brief hints, are added, including some corrections.

(9) In most examples , we begin with a brief explanatory phrase to enhance thereader's understanding.

This textbook can be used for a one- or two-semester course in linear algebra. Atheory oriented one-semester course may cover Chapter 1, Sections 1.1-1.4, 1.6-1.7;Chapter 2 Sections 2.1-2.3; Chapter 3 Sections 3.1-3.6; Chapter 4 Sections 4.1-4.6;Chapter 5 Sections 5.1-5.4; Chapter 6 Sections 6.1-6.2; Chapter 7 Sections 7.1-7.4with possible addition from Sections 1.8, 2.4 or 9.1-9.4. Selected applications areincluded in each chapter as appropriate. For a beginning applied algebra course, aninstructor might include some ofthem in the syllabus at his or her discretion dependingon which area is to be emphasized or considered more interesting to the students.

In definitions , we use bold face for the word being defined, and sometimes an italicor shadowbox to emphasize a sentence or undefined or post-defined terminology.

Preface to the Second Edition vii

Acknowledgement: The authors would like to express our sincere appreciationfor the many opinions and suggestions from the readers of the first edition includingmany of our colleagues at POSTECH. The authors are also indebted to Ki Hang Kimand Fred Roush at Alabama State University and Christoph Dalitz at HochschuleNiederrhein for improving the manuscript and selecting the newly added subjects inthis edition . Our thanks again go to Mrs . Kathleen Roush for grammatical correctionsin the final manuscript, and also to the editing staff of Birkhauser for gladly acceptingthe second edition for publication.

JinHo KwakSungpyo Hong

E-mail: jinkwak@postech.ac.krsungpyo@postech.ac.kr

January 2004, Pohang, South Korea

Preface to the First Edition

Linear algebra is one of the most important subjects in the study of science and engi­neering because of its widespread applications in social or natural science, computerscience, physics , or economics . As one of the most useful courses in undergradu­ate mathematics , it has provided essential tools for industrial scientists. The basicconcepts of linear algebra are vector spaces, linear transformations, matrices anddeterminants, and they serve as an abstract language for stating ideas and solvingproblems .

This book is based on lectures delivered over several years in a sophomore-levellinear algebra course designed for science and engineering students. The primarypurpose of this book is to give a careful presentation of the basic concepts of linearalgebra as a coherent part of mathematics, and to illustrate its power and utility throughapplications to other disciplines . We have tried to emphasize computational skillsalong with mathematical abstractions , which have an integrity and beauty of theirown. The book includes a variety of interesting applications with many examples notonly to help students understand new concepts but also to practice wide applicationsofthe subject to such areas as differential equations, statistics, geometry, and physics.Some of those applications may not be central to the mathematical development andmay be omitted or selected in a syllabus at the discretion of the instructor. Mostbasic concepts and introductory motivations begin with examples in Euclidean spaceor solving a system of linear equations, and are gradually examined from differentpoints of view to derive general principles .

For students who have finished a year of calculus, linear algebra may be the firstcourse in which the subject is developed in an abstract way, and we often find that manystudents struggle with the abstractions and miss the applications . Our experience isthat, to understand the material, students should practice with many problems, whichare sometimes omitted . To encourage repeated practice, we placed in the middle ofthe text not only many examples but also some carefully selected problems, withanswers or helpful hints . We have tried to make this book as easily accessible andclear as possible , but certainly there may be some awkward expressions in severalways. Any criticism or comment from the readers will be appreciated .

x Preface to the First Edition

We are very grateful to many colleagues in Korea, especially to the faculty mem­bers in the mathematics department at Pohang University of Science and Technology(POSTECH), who helped us over the years with various aspects of this book. Fortheir valuable suggestions and comments, we would like to thank the students atPOSTECH, who have used photocopied versions of the text over the past severalyears. We would also like to acknowledge the invaluable assistance we have receivedfrom the teaching assistants who have checked and added some answers or hintsfor the problems and exercises in this book. Our thanks also go to Mrs. KathleenRoush who made this book much more readable with grammatical corrections in thefinal manuscript. Our thanks finally go to the editing staff of Birkhauser for gladlyaccepting our book for publication.

Jin Ho KwakSungpyo Hong

April 1997, Pohang, South Korea

Contents

Preface to the Second Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Preface to the First Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

1 Linear Equations and Matrices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Systems of linear equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Gaussian elimination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Sums and scalar multiplications of matrices. . . . . . . . . . . . . . . . . . . . . 111.4 Products of matrices .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.5 Block matrices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191.6 Inverse matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211.7 Elementary matrices and finding A-I . . . . . . . . . . . . . . . . . . . . . . . . . . 231.8 LDU factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 291.9 Applications..... . .. . . . .. . . .. . .. .... . . . .. .. . . . . ....... . ... . 34

1.9.1 Cryptography.. . . .. . . . . . . . .. . .. . .. .. .. . .. . .. . . .. .. .. 341.9.2 Electrical network 361.9.3 Leontief model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

1.10 Exercises 40

2 Determinants. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 452.1 Basic properties of the determinant. . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.2 Existence and uniqueness of the determinant. . . . . . . . . . . . . . . . . . .. 502.3 Cofactor expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 562.4 Cramer's rule. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 612.5 Applications .... . ....... ... . . . .. . ... ............. ... ... .... 64

2.5.1 Miscellaneous examples for determinants. . . . . . . . . . . . . . .. 642.5.2 Area and volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 67

2.6 Exercises ............. ..... . . .. . . .. . ...... .... .. .. . .. .. . . . 72

xii Contents

3 Vector Spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 753.1 The n-space jRn and vector spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 753.2 Subspaces. . . . .. . . . ... . . .. . . . .. . . . . . ... . ... ... . . . . ... .... .. 793.3 Bases. . . .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 823.4 Dimensions . ... . ... .. .. ... .. .. . .. .. . . . . .... . .. ... . . . ... . .. 883.5 Rowand columnspaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 913.6 Rank and nullity 963.7 Bases for subspaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 1003.8 Invertibility.... .. ...... ... .. ... . .. . . ... . . . . . . . . . . . . . . . . . . . 1063.9 Applications 108

3.9.1 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1083.9.2 The Wronskian 110

3.10 Exercises 112

4 Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1174.1 Basic propertiesof linear transformations 1174.2 Invertiblelinear transformations . . . . . . . . . . . . . . . . . . . . . 1224.3 Matrices of linear transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . .. 1264.4 Vector spaces of linear transformations . . . . . . . . . . . . . . . . . . . . . . . . . 1314.5 Change of bases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1344.6 Similarity. . . . .. .. . . . . . . . . . .. . . . . . .. .... .. . . . ... .. . . . ... . .. 1384.7. Applications 143

4.7.1 Dual spaces and adjoint " 1434.7.2 Computer graphics 148

4.8 Exercises ... . . . . . .. . ... . . . .. . .. . . . . . . . . . . . .. . .. . . . . . . . . . .. 152

5 Inner Product Spaces 1575.1 Dot products and inner products 1575.2 The lengths and angles of vectors.. . . . .. . .. .. . . .. . . .. .. . . .. . .. 1605.3 Matrix representations of inner products 1635.4 Gram-Schmidt orthogonalization 1645.5 Projections. .... . .. . . ... .. . . ... .... .... ................. . . . 1685.6 Orthogonalprojections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1705.7 Relations of fundamental subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 1755.8 Orthogonalmatrices and isometries 1775.9 Applications 181

5.9.1 Least squares solutions 1815.9.2 Polynomial approximations 1865.9.3 Orthogonalprojectionmatrices. . . . . . . . . . . . . . . . . . . . . . . . . 190

5.10 Exercises 196

Contents xiii

6 Diagonalization 2016.1 Eigenvalues and eigenvectors 2016.2 Diagonalization of matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2076.3 Applications 212

6.3.1 Linear recurrence relations . . . . . . . . . .. 2126.3.2 Linear difference equations 2216.3.3 Linear differential equations I . . . . . . . . . . . . . . . . . . . . . . . . . . 226

6.4 Exponential matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 2326.5 Applications continued . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 235

6.5.1 Linear differential equations II 2356.6 Diagonalization of linear transformations . . . . . . . . . . . . . . . . . . . . . . . 2406.7 Exercises . .. .. . .. .. .... . ... .... .... .. ....... . .. .. . ....... . 242

7 Complex Vector Spaces 2477.1 The n-space en and complex vector spaces " 2477.2 Hermitian and unitary matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 2547.3 Unitarily diagonalizable matrices 2587.4 Normal matrices 2627.5 Application . ...... .. .... . ........ ..... .............. .. .. .. . 265

7.5.1 The spectral theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 2657.6 Exercises.. ...... .. .... . .. ..... . ... .... .... . . ........ ... . . 269

8 Jordan Canonical Forms 2738.1 Basic properties of Jordan canonical forms . . . . . . . . . . . . . . .. 2738.2 Generalized eigenvectors " 2818.3 The power Ak and the exponential eA .• •. •• • • • •. ••• •• •••. •.•. •. 2898.4 Cayley-Hamilton theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 2948.5 The minimal polynomial of a matrix " 2998.6 Applications. .... .. ... ...... .... ...... .. ..... .. .. .. .. ...... 302

8.6.1 The power matrix Ak again 3028.6.2 The exponential matrix eA again 3068.6.3 Linear difference equations again . . . . . . . . . . . . . . . . . . . . . .. 3098.6.4 Linear differential equations again. . . . . . . . . . . . . . . . . . . . . . 310

8.7 Exercises 315

9 Quadratic Forms 3199.1 Basic properties of quadratic forms " 3199.2 Diagonalization of quadratic forms 3249.3 A classification of level surfaces 3279.4 Characterizations of definite forms " 3329.5 Congruence relation 3359.6 Bilinear and Hermitian forms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3399.7 Diagonalization of bilinear or Hermitian forms 3429.8 Applications 348

9.8.1 Extrema of real-valued functions on jRn • • .• • • • • • •• . • • • . • 348

xiv Contents

9.8.2 Constrained quadratic optimization 3539.9 Exercises.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

SelectedAnswersand mots 361

Bibliography 383

Index 385

Linear Algebra