toomas linear algebra book 185 p

P art IAPPLICA TIONS OF LINEARALGEBRAINTR ODUCTIONThis course /\Applications of Linear Algebra/" is based on the lecturesgiv en b y the author to p ostgraduate studen ts at T allinn T ec hnical Univ er/-sit y /. Our aim w as to acquain t the studen ts with the linear algebra pac k agesLINP A CK/, EISP A CK and LAP A CK/, and with the theoretical fundamen talsof the parts of the pac k ages MA TLAB/, MAPLE/, MA THCAD and MA TH/-EMA TICA related to linear algebra/. W e ha v e tried to explain the linearalgebra metho ds whic h form the basis for the computing metho ds used inthe pac k ages/. W e w ould lik e to stress that the aim of the course is not tow ork out concrete computing algorithms but to learn ab out the basic ideasrelated to these algorithms/. It will b e assumed that the reader is acquain tedwith the basic ideas of algebra/.The author w ould lik e to thank Asso c/. Prof/. Ellen Redi /(T allinn P ed/-agogical Univ ersit y/) whose help in the impro v emen t of the presen ted math/-erial b oth in its con ten ts and its form has b een enormous/. Man y of theexamples and problems w ere prepared b y studen ts Kristiina Kr / uspan/, KadriMikk/, Reena Prin ts /(T allinn P edagogical Univ ersit y/)/, Andrei Filono v/, DmitriTseluik o /(T artu Univ ersit y/) Juhan/-P eep Ernits and Heiki Hiisj/ arv /(T allinnT ec hnical Univ ersit y/) within the framew ork of the TEMPUS/-pro ject duringtheir sta y at T amp ere Univ ersit y of T ec hnology in June/, /1/9/9/7/.The n um be r s of their examples and problems are mark ed b y an asterisk/\/*/"/.The matherial is based on the monographs of G/.H/.Golub and C/.F/.V anLoan /(/1/9/9/6/)/, and G/.Strang /(/1/9/8/8/)/.I hop e that the course will help the reader in terested in applications oflinear algebra more to use the linear algebra pac k ages more e/ectiv ely /.Author/./1 /1 FUND AMENT A TIONS OF LINEAR AL/-GEBRA/1/./1 V ectors/1/./1/./1 V ector SpacesOne of the fundamen tal concepts of linear algebra is that of v ector space/.A t the same time it is one of the more often used concepts of algebraicstructure in mo dern mathematics/. F or example/, man y function sets studiedin mathematical analysis are with resp ect to their algebraic prop erties v ectorspaces/. In analysis the notion /\linear space/" is used instead of the notion/\v ector space/"/.De/nition /1/./1/./1 /: As e t X is called a ve ctor sp ac e over the numb er /eld K /;;if to ev ery pair /( x /;; y /) of elemen ts of X there corresp onds as u m x /+ y /2 X /;; /,and to ev ery pair /( //;; x /) /;; where / /2 K and x /2 X /;; /, there corresp onds anelemen t / x /2 X /, with the prop erties /1/-/8/:/1/. x /+ y /= y /+ x /(comm utabilit y of addition/)/;;/2/. x /+/( y /+ z /)/= /( x /+ y /)/+ z /(asso ciativit y of addition/)/;;/3/. /9 /0 /2 X /: /0 /+ x /= x /(existence of n ull elemen t/)/;;/4/. /8 x /2 X /) /9 /; x /2 X /: x /+/( /; x /) /= /0 /(existence of the in v erseelemen t/)/;;/5/. /1 /
x /= x /(unitarism/)/;;/6/. / /( / x /)/= /( // /) x /(asso ciativit y with resp ect to n um be r m ultiplication/)/;;/7/. / /( x /+ y /)/= / x /+ / y /(distributivit y with resp ect to v ector addition/)/;;/8/. /( / /+ / /) x /= / x /+ / x /(distributivit y with resp ect to n um b er addition/)/.The prop erties /1/-/8 are called the v ector space axioms/. Axioms /1/-/4 sho wthat X is a comm utativ e group or an Ab elian group with resp ect to v ectoraddition/. The second corresp ondence is called m ultiplication of the v ector b ya n um be r /, and it satis/es axioms /5/-/8/. Elemen ts of a v ector space are calledv ectors/. If K /= R /, then one sp eaks of a real v ector space/, and if K /= C /,then of a complex v ector space/. Instead of the notion /\v ector space/" w e shalluse the abbreviativ e /\space/"/./2 Example /1/./1/./1/. Let us consider the set of all n / /1 /; matrices with realelemen ts/:X /= f x /: x /= /2/6/6/4 //1/./././n /3/7/7/5 /^ /i /2 R g /:The sum of t w o matrices w e de/ne in the usual w a y b y the addition of thecorresp onding elemen ts/. By m ultiplying the matrix b y a real n um be r / w em ultiply all elemen ts of the matrix b y this n um be r /. The simple c hec k willsho w that conditions /1/-/8 are satis/ed/. F or example/, let us c hec k conditions/3 and /4/. W e construct/0 /= /2/6/6/4 /0/./. /./0 /3/7/7/5 /;; /; x /= /2/6/6/4 /; //1/./. /./; /n /3/7/7/5 /:As/0 /+ x /= /2/6/6/4 /0/./. /./0 /3/7/7/5 /+ /2/6/6/4 //1/./. /./n /3/7/7/5 /= /2/6/6/4 /0/+ //1/./. /./0/+ /n /3/7/7/5 /= /2/6/6/4 //1/./. /./n /3/7/7/5 /= x /;;the elemen t /0 satis/es condition /3 for arbitrary x /2 X /, and th us it is then ull elemen t of the space X /. F or the elemen t /; xx /+/( /; x /)/= /2/6/6/4 //1/./. /./n /3/7/7/5 /+ /2/6/6/4 /; //1/./. /./; /n /3/7/7/5 /= /2/6/6/4 //1 /; //1/./. /./n /; /n /3/7/7/5 /= /2/6/6/4 /0/./. /./0 /3/7/7/5 /= /0 /;;i/.e/./, condition /4 is satis/ed/. Mak e sure of the v alitidy of the remainingconditions /1/-/2 and /5/-/8/.The v ector space in example /1/./1/./1 is called an n /- dimensional r e al arith/-metic al sp ac e or in short R n/. Declaring the v ector x of the space R nw e oftenuse the transp osed matrixx /= h//1 /:/:/: /n iT/:In this presen tation w e often use punctuation marks /(comma/, semicolon/) toseparate the comp onen ts of the v ector/, for examplex /= h//1 /;; /:/:/: /;;/n iT/:/3 Example /1/./1/./1/. /Let U b e a set that consists of all pairs of real n um be r sa /= /( //1 /;;//2 /) /;; b /=/( //1 /;;//2 /) /;; /:/:/: W e de/ne addition and m ultiplication b y ascalar in U as follo ws/:a /+ b /=/( /( / /3/1 /+ / /3/1 /) /1 /= /3/;; /( / /3/2 /+ / /3/2 /) /1 /= /3/) /;;/ a /=/( ///1 /;;/ //2 /) /:Is the set U a v ector space/?Prop osition /1/./1/./1/. Let X be a v ector space/. F or arbitrary v ectorsx /;; y /2 X and n um be r / /2 K the follo wing assertions and equalities are v alid/:/ the n ull v ector /0 of the v ector space X is unique/;;/ the in v erse v ector /; x of eac h x /2 X is unique/;;/ the uniqueness of the in v erse v ector allo ws to de/ne the op eration ofsubtraction b yx /; y def/= x /+/( /; y /)/;;/ x /= y /, x /; y /= /0 /;;/ /0 x /= /0 /8 x /2 X /;;/ / /0 /= /0 /8 / /2 K /;;/ /( /; /1/) x /= /; x /;;/ / x /= /0 /, /( / /=/0 /_ x /= /0 /) /:Become con vinced of the trueness of these assertions/! /2Example /1/./1/./2/. Let us consider the set of all /( m / n /) /; matrices withcomplex elemen ts/. The sum of these matrices will b e de/ned b y the additionof the corresp onding elemen ts of the matrices/. By m ultiplying the matrix b ya complex n um be r / one will m ultiply b y this n um b er all the elemen ts of thematrix/. W el e a v e the c hec k that all conditions /1/-/8 are satis/ed to the reader/.This v ector space o v er the complex n um b er /eld C will b e denoted C m / n/: Ifw e con/ne ourselv es to real matrices/, then w e shall get a v ector space R m / no v er the n um be r /eld R /: The space C m / /1will be iden ti/ed with the spaceC mand the space R m / /1with the space R m/:/4 Example /1/./1/./3/. The set F /[ //;; / /] of all functions x /: /[ //;; / /] /! R is av ector space /(pro v e/!/) o v er the n um be r /eld R if/( x /+ y /)/( t /) def/= x /( t /)/+ y /( t /) /8 t /2 /[ //;; / /]and/( / x /)/( t /) def/= / x /( t /) /8 t /2 /[ //;; / /] /:/1/./1/./2 Subspaces of the V ector SpaceDe/nition /1/./2/./1/. The set W of v ectors of the v ector space X /(o v er the/eld K /) that is a v ector space will resp ect to v ector addition and m ultiplica/-tion b y a n um be r de/ned in the v ector space X /, is called a subsp ac e of thev ector space X and denoted W / X /:Prop osition /1/./2/./1 /: The set W of v ectors of the v ector space X is asubspace of the v ector space X i/ for eac h t w o v ectors x /;; y /2 W and eac hn um be r / /2 K v ectors x /+ y and / x b elong to the set W /.Pr o of/. Necessit yi s o b vious/. T o pro v e su/ciency /,w eh a v e to sho w that inour case conditions /1/-/8 for a v ector space are satis/ed/. Let us c hec k condition/1/. Let x /;; y /2 W / X /: By assumption/, x /+ y /2 W / X /. As X is a v ectorspace/, then for X axiom /1 is satis/ed/, and then x /+ y /= y /+ x /. Therefore/,for W axiom /1 is satis/ed/, to o/. Let us test the v alidit y of condition /4/.Let x /2 W / X /: By assumption/, /( /; /1/) x /2 W / X /: On the other hand/, b yprep osition /1/, in X the equalit y/( /; /1/) x /= /; x /: holds/. Hence the in v erse v ector/; x b elongs to set W with the v ector x /, i/.e/./, condition /4 is satis/ed/. Pro v eb y y ourselv es the v alidit y of conditions /2/, /3 and /5/-/8/. /2Example /1/./2/./1/. The v ector space C /[ //;; / /]o v er R of all functions con ti/-nouos on /[ //;; / /] /(example /1/./1/./3/) is a subspace of v ector space F /[ //;; / /] /: As thesum of t w o functions con tinouos on the in terv al/, and the pro duct of suc h afunction b ya n um b er are functions con tinouos on this in terv al/, b y prop osition/1/./2/./1/, C /[ //;; / /] is a subspace of the v ector space F /[ //;; / /] /:Example /1/./2/./2/. Let Pn be the set of all p olynomials a/0 t k/+ a/1 t k /; /1/+/:/:/: /+ ak /; /1 t /+ ak /= x /( k / n /) of at most degree n with real co e/cien ts/. W ede/ne addition of t w o p olynomials and m ultiplication of a p olynomial b y areal n um be r in the usual w a y /. As a result/, w e get the v ector space Pn ofp olynomials of at most degree n/: If w e denote b y Pn /[ //;; / /]t h e v ector space of/5 p olynomials of at most degree n de/ned on the in terv al /[ //;; / /]/, then Pn /[ //;; / /]will b e a subspace of the v ector space C /[ //;; / /] /:Example /1/./2/./3/. /Let us sho w that the set H /= /(/"a b/0 c /#/: a/;; b/;; c /2 R /)is a subspace of the matrix v ector space R /2 / /2/:The set H is closed with resp ect to additism and m ultiplication b y scalarsince/"a b/0 c /#/+ /"d e/0 f /#/= /"a /+ d b /+ e/0 c /+ f /#and/ /"a b/0 c /#/= /"/a /b/0 /c /#/:Th us the set H is a subspace of the matrix v ector space R /2 / /2/:Problem /1/./2/./1/. /Pro v e that the set of all symmetric matrices form asubspace of the v ector space of all square matrices R n / n/:Prop osition /1/./2/./2/. If S/1 /;;/:/:/: /;; Sk are subspaces of the v ector space X /,then the in tersection S /= S/1 /\ S/2 /\ /:/:/: /\ Sk of the subspaces is a subspace ofthe v ector space X /:Pro v e/! /2Prop osition /1/./2/./3/. If S/1 /;; /
/
/
/;; Sk are subspaces of the space X andS /= f x/1 /+ x/2 /+ /:/:/: /+ xk /: xi /2 Si /(i/=/1 /: k/) gis the sum of these subspaces/, then S is a subspace of X /:De/nition /1/./2/./2/. If eac h x /2 S can be expressed uniquely in the formx /= x/1 /+ x/2 /+ /:/:/: /+ xk /( xi /2 Si /) /;; then w e sa y that S is the dir e ct sum ofsubsp ac es Si /;; and it denoted S /= S/1 / S/2 / /
/
/
/ Sk /:De/nition /1/./2/./3/. Eac h elemen t of the space X that can be expressedas //1 x/1 /+ /:/:/: /+ /n xn /;; where /i /2 K /;; is called a line ar c ombination of theelemen ts x/1 /;;/:/:/: /;; xn of the v ector space X /(o v er the /eld K /)/.De/nition /1/./2/./4/. The set of all p ossible linear com bination of the set Zis called the sp an of the set Z / X /:Example /1/./2/./4/. Let X /= R /3and Z /= f /[ /1/;; /1/;; /0/] T/;; /[/1/;; /; /1/;; /0/] Tg /: Thenspan Z /= f /[ / /;; / /;;/0 /] T/: //;; / /2 R g /: Pro v e/!/6 Prop osition /1/./2/./4/. The set span Z of the set Z / X is the least subspacethat con tains the set Z/:Pr o of/. First/, let us pro v e that span Z is a subspace of the space X /. Itis su/cien t/, b y prop osition /1/./2/./1/, to sho w that span Z is closed with resp ectto v ector addition and m ultiplication of the v ector b y a n um b er/:x /;; y /2 span Z /, x /= nXi /=/1 /i ui /^ y /= mXj /=/1 /j vj /^ /i /;;/j /2 K /^ ui /;; vj /2 Z /)x /+ y /= nXi /=/1 /i ui /+ mXj /=/1 /j vj /^ /i /;;/j /2 K /^ ui /;; vj /2 Z /, x /+ y /2 span Z /;;/ /2 K /^ x /2 span Z /, / /2 K /^ x /= nXi /=/1 /i ui /^ ui /2 Z /^ /i /;;/ /2 K /)/ x /= / nXi /=/1 /i ui /= nXi /=/1 /( /i /i /) ui /= nXi /=/1 /i ui /^ /i /2 K /^ ui /2 Z /, / x /2 span Z /:Th us/, span Z is a subspace of the space X /. Let us sho w that span Z isthe least subspace of the space X that con tains the set Z/: Let Y be somesubspace of the space X for whic h Z / Y /: As Z / Y and Y is a subspace/,the arbitrary linear com bination of the elemen ts os the set Z b elongs to thesubspace Y /: Therefore/, span Z as the set of all suc h linear com binationsb elongs to the space Y /: /2Corollary /1/./2/./1/. A subset W of the v ector space X is a subspace i/ itcoincides with its span/, i/.e/./, W / X /, W /= span W /:Problem /1/./2/./2/. /Do es the v ector d /= h/8 /7 /4 iTb elong to the sub/-space span f a /;; b /;; c g /;; /, whena /= h/1 /; /1 /0 iT/;; b /= h/2 /3 /1 iT/;; c /= h/6 /9 /3 iT/?/7 /1/./1/./3 Linear Dep endence of V ectors/. Basis of the V ector Space/.De/nition /1/./3/./1/. A set of v ectorsf x/1 /;;/:/:/: /;; xk gin the v ector space X /(o v er the /eld K /)i s said to be line arly dep endent if/9 //1 /;;/:/:/: /;;/k /2 K /: j //1 j /+ /:/:/: /+ j /k j /6/=/0 /^ //1 x/1 /+ /:/:/: /+ /k xk /= /0 /:De/nition /1/./3/./2/. A set of v ectors in the space X /(o v er the /eld K /) issaid to be line arly indep endent if it is not linearly dep enden t/.Example /1/./3/./1/. /Let us c hec k if the set U /= f /1/+ x/;; x /+ x /2/;; /1/+ x /2g islinearly indep enden t in the v ector space Pn /( n / /2/) of all p olynomials of atmost degree n with real co e/cien ts/.Let us consider the equalit y/ /(/1 /+ x /)/+ / /( x /+ x /2/)/+ / /(/1 /+ x /2/)/= /0 /:It is w ell/-kno wn in algebra that a p olynomial is iden tically n ull i/ all itsco e/cien ts are zeros/. Th us w e get the system/8/>/</>/: / /+ / /=/0/ /+ / /=/0/ /+ / /=/0 /:This system has only a trivial solution/. The set U is linearly indep enden t/.Problem /1/./3/./1/. /Pro v e that eac h set of v ectors that con tains the n ullv ector is linearly dep enden t/.Problem /1/./3/./2/. /Pro v e that if the column/-v ectors of a determinan t arelinearly dep enden t/, then the determinan t equals /0/.De/nition /1/./3/./3/. A subset V /= f xi/1 /;;/:/:/: /;; xik g of the set U /= f x/1 /;;/:/:/: /;; xn gof v ectors of the v ector space X is called a maximal line arly indep endent sub/-set if V is linearly indep enden t and it is not a prop er subset of an y linearlyindep enden t subset of the set U /.Prop osition /1/./3/./1/. If V is a maximal linearly indep enden t subset of theset U/;; then span U /= span V /:/8 Pr o of/. As V / U/;; s p a n V / span U /;; b y the de/nition of the span/. T o pro v eour assertion/, w eh a v et o s h o w that span V / span U/: Let/, b ya n tithesis/, exista v ector x of the subspace span U that do es not b elong to the subspacespan V /: Th us/, the v ector x cannot be expressed as a linear com bination ofv ectors of V but can be expressed as a linear com bination of v ectors of U/;;when at least one v ector xj /2 U is used/, at whic h xj /= /2 V and xj is notexpressable as a linear com bination of v ectors of V/: Set V /[f xj g / U islinearly indep enden t and con tains the set V as a prop er subset/. Hence V isnot the maximal linearly indep enden t subset/. W e ha v e got a con tradictionto the assumption/. Th us span V / span U /;; Q/.E/.D/. /2De/nition /1/./3/./4/. A set B /= f xi gi /2 I of v ectors of the v ector space Xis called a b asis of the v ector space X if B is linearly indep enden t and eac hv ector x of the space X can be expressed as a linear com bination of v ectorsof the set B/;; x /= Pi /2 I /i xi /, where co e/tien ts /i /( i /= /1 /: n /) are calledc o or dinates of the v ector x relativ et ot h e basis B/:De/nition /1/./3/./5/. If the n um be r of v ectors in the basis B of the v ectorspace X /;; i/.e/./, the n um b er of elemen ts of the set I/;; is /nite/, then this n um be ris called the dimension of the ve ctor sp ac e X and denoted dim X /;; and thespace X is called a /nite/-dimensional or a /nite/-dimensional ve ctor sp ac e /. Ifthe n um be r of v ectors in the basis B of the v ector space X is in/nite/, thenthe v ector space X is called in/nite/-dimensional or an in/nite/-dimensionalve ctor sp ac e /.Prop osition /1/./3/./2/. A subset B of the v ectors of the v ector space X isa basis of the space i/ it is the maximal linearly indep enden t subset/.Example /1/./3/./2/. V ectorsek /= /[/0/;; /0/;; /:/:/: /;;/0k /; /1 ze r o s /;;/1 /;; /0 /;; /:/:/: /;;/0n /; k z er os /] T/(k /= /1/: n/)form a basis in space R n/: Let us c hec k the v alidit y of the conditions inde/nition /1/./3/./4/. AsnXk /=/1 /k ek /= /0 /, /[ //1 /;;/:/:/: /;;/n /] T/= /[/0/;; /:/:/: /;;/0 /] T/, nXk /=/1 j /k j /=/0 /;;the v ector system f ek gk /=/1/: n is linearly indep enden t/, and/, due to/[ //1 /;;/:/:/: /;;/n /] T/= nXk /=/1 /k ek /;;/9 an arbitrary v ector of the space R ncan b e expressed as a linear com binationof v ectors ek /:Problem /1/./3/./3/. V ector system/(/"/1 /0/0 /0 /#/;; /"/0 /1/0 /0 /#/;; /"/0 /0/1 /0 /#/;; /"/0 /0/0 /1 /#/)forms a basis in space R /2 / /2/:Example /1/./3/./3/. V ector system f /1/;; t /;; t /2/;; /:/:/: /;; t ng forms a basis in v ectorspace Pn of p olynomials of at most degree n/: T ruely /, the set f /1/;; t /;; t /2/;; /:/:/: /;; t ngis linearly indep enden t sincex /= a/0 t n/+ a/1 t n /; /1/+ /:/:/: /+ an /; /1 t /+ an /= /0 /) ak /=/0 /( k /=/1/: n/)and eac hv ector of the space Pn /(i/.e/./, arbitrary p olynomial of at most degreen /) can be expressed in the formx /= a/0 t n/+ a/1 t n /; /1/+ /:/:/: /+ an /; /1 t /+ an /:De/nition /1/./3/./6/. Tw o v ector spaces X and X /0are called isomorphic /,if there exist a one/-to/-one corresp ondence be t w een the spaces /' /: X /! X /0/;;suc h that/1/) /8 x /;; y /2 X /' /( x /+ y /)/= /' /( x /)/+ /' /( y /)/;;/2/) /8 x /2 X /;; /8 / /2 K /' /( / x /)/= //' /( x /) /:Prop osition /1/./3/./3/. All v ector spaces /(o v er the same n um be r / e l d K /)o fthe same dimension are isomorphic/./1/./1/./4 Scalar Pro ductDe/nition /1/./4/./1/. A v ector space X o v er the /eld K is called a sp ac ewith sc alar pr o duct if to eac h pair of elemen ts x /;; y /2 X there corresp onds acertain n um be r h x /;; y i/2 K /;; called the sc alar pr o duct of the ve ctors x and y /;;suc h that follo wing condition /(the axioms of scalar pro duct/) are satis/ed/:/1/. h x /;; x i/ /0/;; h x /;; x i /=/0 /) x /= /0 /;;/1/0 /2/. h x /;; y i /= h y /;; x i /;; when h y /;; x i is the conjugate complex n um be r ofh x /;; y i /;;/3/. h x /+ y /;; z i /= h x /;; z i /+ h y /+ z i /(additivit y with resp ect to the /rst fac/-tor/)/;;/4/. h / x /;; y i /= / h x /;; y i /(homogeneit y with resp ect to the /rst factor/)/.If X is a v ector space o v er R /, then/, b y the de/nition/, h x /;; y i/2 R /;; andcondition /1 acquires the form h x /;; y i /= h y /;; x i /, i/.e/./, in this case scalar pro ductis comm utativ e/.Example /1/./4/./1/. Let us de/ne in C nthe scalar pro duct of v ectorsx /= h//1 /
/
/
/n iT/^ y /= h//1 /
/
/
/n iTb y the form ulah x /;; y i /= nXk /=/1 /k /k /.Let us c hec kt h e v alidit y of conditions /1/-/4/: h x /;; x i /= Pnk /=/1 /k /k /= Pnk /=/1 j /k j /2//0 /;;h x /;; x i /= Pnk /=/1 j /k j /2/=/0 /) /k /=/0 /( k /=/1 /: n /) /, x /= /0 /;;h x /;; y i /= Pnk /=/1 /k /k /= Pnk /=/1 /k /k /= Pnk /=/1 /k /k /= h y /;; x i /;;h x /+ y /;; z i /= Pnk /=/1 /( /k /+ /k /) /&k /= Pnk /=/1 /k /&k /+ Pnk /=/1 /k /&k /= h x /;; z i /+ h y /;; z i /;;h / x /;; y i /= Pnk /=/1 //k /k /= / Pnk /=/1 /k /k /= / h x /;; y i /:Example /1/./4/./2/. Let us consider the v ector space L/2 /[ //;; / /] of all functionsin tegrable /(in Leb esque/'s sense/) on the in terv al /[ //;; / /] /: W e de/ne the scalarpro duct for suc h functions b y the form ulah x /;; y i /= Z// x /( t /) y /( t /) dt/:V erify that all the axioms /1/-/4 of scalar pro duct are satis/ed/.Prop osition /1/./4/./1/. Scalar pro duct h x /;; y i has the follo wing prop erties/:/1/. h x /;; y /+ z i /= h x /;; y i /+ h x /;; z i /(additivit y with resp ect to the secondfactor/)/;;/2/. h x /;;/ y i /= / h x /;; y i /(conjugate homogeneit y with resp ect to the secondfactor/)/;;/3/. h x /;; /0 i /= h /0 /;; y i /=/0 /8 x /;; y /2 X /;;/1/1 /4/. h / x /;;/ y i /= j / j /2h x /;; y i /:Let us pr ove these assertions/:h x /;; y /+ z i /= h y /+ z /;; x i /= h y /;; x i /+ h z /;; x i /= h y /;; x i /+ h z /;; x i /= h x /;; y i /+ h x /;; z i /;;h x /;;/ y i /= h / y /;; x i /= / h y /;; x i /= / h y /;; x i /= / h x /;; y i /;; h x /;; /0 i /= h x /;; /0 x i /= /0 h x /;; x i /=/0/;;h / x /;;/ y i /= / / h x /;; y i /= j / j /2h x /;; y i /: /2Prop osition /1/./4/./2 /(Cauc h y/-Sc h w artz inequalit y/)/. F or arbitrary v ectorsx and y of the v ector space with scalar pro duct X it holds the inequalit yjh x /;; y i j/ q h x /;; x i q h y /;; y i /:Pr o of/. If h x /;; y i /= /0/, then/, b y the de/nition of the scalar pro duct /(condition/1/) the inequalit y holds/. No w let us consider the case h x /;; y i /6/=/0 /: W e de/ne anauxiliary function/' /( / /)/= h x /+ / h x /;; y i y /;; x /+ / h x /;; y i y i /:As for / /2 R/' /( / /)/= h x /;; x i /+ / h x /;; y ih x /;; y i /+ / h x /;; y ih y /;; x i /+ / /2j h x /;; y i j /2h y /;; y i /=/= / /2jh x /;; y i j /2h y /;; y i /+ /2 / j h x /;; y i j /2/+ h x /;; x i/ /0 /8 / /2 R /,/,j h x /;; y i j /4/;j h x /;; y i j /2h x /;; x ih y /;; y i/ /0 /:The last inequalit y is equiv alen t to the inequalit y j h x /;; y i j /2/h x /;; x ih y /;; y i /;; andthis /| to the Cauc h y/-Sc h w artz inequalit y /. The Cauc h y/-Sc h w artz inequal/-it y mak es it p ossible to de/ne the angle be t w een t w o v ectors b y the scalarpro duct/.De/nition /1/./4/./2/. The angle be t w een arbitrary v ectors x and y of thev ector space with scalar pro duct X is de/ned b y the form ulacos/( dx /;; y /)/= h x /;; y i /= /( q h x /;; x i q h y /;; y i /) /:Problem /1/./4/./1/. /Sho w that for eac h t w o complex v ectors x and y theequalit yh x /;; y i /= h x /;; y i /:holds/.Problem /1/./4/./2/. /The scalar pro duct in the v ector space Pn /[ //;; / /] of p oly/-nomials of at most degree n with real co e/cien ts on /[ //;; / /] is de/ned b y theform ulah x /;; y i /= Z// x /( t /) y /( t /) dt/:/1/2 Find the angle b et w een the p olynomials x /= t /; /1a n d y /= t /2/+/1 /:/1/./1/./5 Norm of a V ectorDe/nition /1/./5/./1/. Av ector space X /(o v er the n um be r / e l d K /) is calleda norme d sp ac e/, if to eac h v ector x /2 X there corresp onds a certain non/-negativ e real n um be r k x k /;; called the norm of the ve ctor /, suc h that the fol/-lo wing conditions are satis/ed/:/1/. k x k /=/0 /, x /= /0 /( iden tit y axiom/)/;;/2/. k / x k /= j / jk x k /(homogeneit y axiom/)/;;/3/. k x /+ y k/k x k /+ k y k /(triangle inequalit y/)/.De/nition /1/./5/./2/. The distanc e / /( x /;; y /) b etwe en two ve ctors in the normedspace X is de/ned b y the form ula / /( x /;; y /)/= k x /; y k /:Prop osition /1/./5/./1 /(H/ older inequalit y/) /. If /1 /<p /< /1 /;; /1 /=p /+/1 /=q /=/1 /;;x /= h//1 /
/
/
/n iT/2 C n/^ y /= h//1 /
/
/
/n iT/2 C n/;;thennXk /=/1 j /k /k j/ / nXk /=/1 j /k j p /!/1 /=p / nXk /=/1 j /k j q /!/1 /=q/:Pr o of/. See E/.Oja/, P /.Oja /(/1/9/9/1/, pp/. /1/1/-/1/2/)/.Prop osition /1/./5/./2 /(Mink o wski inequalit y/)/. If /1 / p/< /1 /;;x /= h//1 /
/
/
/n iT/2 C n/^ y /= h//1 /
/
/
/n iT/2 C n/;;then/ nXk /=/1 j /k /+ /k j p /!/1 /=p/ / nXk /=/1 j /k j p /!/1 /=p/+ / nXk /=/1 j /k j p /!/1 /=p/:Pr o of/. See E/.Oja/, P /.Oja /(/1/9/9/1/, pp/. /1/0/-/1/1/) /.Example /1/./5/./1/. One de/nes in C nthe p /-norm /(/1 / p / /1 /) of thev ector x b y the form ulask x kp /=/( j //1 j p/+ /:/:/: /+ j /n j p/) /1 /=p/(/1 / p/< /1 /) /;;/1/3 k x k/1 /= max/1 / k / n j /k j /:Let us v erify that the p /-norm /(/1 / p /< /1 /) satis/es the conditions /1/-/3 inde/nition /1/./5/./1/:k x kp /=/( j //1 j p/+ /:/:/: /+ j /n j p/) /1 /=p/=/0 /, /k /=/0 /(/1 / k / n /) /, x /= /0 /;;k / x kp /=/( j ///1 j p/+ /:/:/: /+ j //n j p/) /1 /=p/=/[ j / j p/( j //1 j p/+ /:/:/: /+ j /n j p/)/] /1 /=p/=/= j / j /( j //1 j p/+ /:/:/: /+ j /n j p/) /1 /=p/= j / jk x kp /;;using the Mink o wski inequalit y /,w e getk x /+ y kp /= / nXk /=/1 j /k /+ /k j p /!/1 /=p/ / nXk /=/1 j /k j p /!/1 /=p/+ / nXk /=/1 j /k j p /!/1 /=p/= k x kp /+ jj y jjp /:V erify conditions /1/-/3 in case of the norm k/
k/1 /!The most often used p /-norms are/:k x k/1 /= j //1 j /+ /:/:/: /+ j /n j /;;k x k/2 /=/( j //1 j /2/+ /:/:/: /+ j /n j /2/) /1 /= /2/;;k x k/1 /= max/1 / k / n j /k j /:Problem /1/./5/./1/. /Let b e giv en the v ectors u /= h/1 /; /1 /3 iTja v /= h/0 /3 /2 iT/:Findk u /+ v k/1 /;; k u /+ v k/2 /;; k u /+ v k/1 /;; k u k/1 /+ k v k/1 /;; k u k/2 /+ k v k/2 /;; k u k/1 /+ k v k/1 /;;k/; /5 u k/1 /+/5 k v k/1 /;; k/; /5 u k/2 /+/5 k v k/2 /;; k/; /5 u k/1 /+/5 k v k/1 /:Prop osition /1/./5/./3/. All the p /-norms of the space C nare equiv alen t/, i/.e/./,if k/
k/ and k/
k/ are the p /-norms of the space C n/, then there exist p ositiv econstan ts c/1 and c/2 /;; suc h thatc/1 k x k/ /k x k/ / c/2 k x k/ /8 x /2 C n/:A t the same timek x k/2 /k x k/1 / p n k x k/2 /;;k x k/1 /k x k/2 / p n k x k/1 /;;k x k/1 /k x k/1 / n k x k/1 /:/1/4 Let us pr ove the last three assertions/:k x k/2 /=/( j //1 j /2/+ /:/:/: /+ j /n j /2/) /1 /= /2/ /0/@ nXi /=/1 nXj /=/1 j /i jj /j j /1A /1 /= /2/=/=/( /( nXk /=/1 j /i j /) /2/) /1 /= /2/= k x k/1 /;;Using the H/ older inequalit y /,w e get in case p /= q /= /2 thatk x k/1 /= j //1 j /+ /:/:/: /+ j /n j /=/1 /
j //1 j /+ /:/:/: /+/1 /
j /n j /= j /1 /
//1 j /+ /:/:/: /+ j /1 /
/n j// /(/1 /2/+ /:/:/: /+/1 /2/) /1 /= /2/( / /2/1 /+ /:/:/: /+ / /2n /) /1 /= /2/= p n k x k/2 /;;k x k/1 /= max/1 / k / n j /k j /= /(/( max/1 / k / n j /k j /) /2/) /1 /= /2/ /( / /2/1 /+ /:/:/: /+ / /2n /) /1 /= /2/= k x k/2 /;;k x k/2 /=/( / /2/1 /+ /:/:/: /+ / /2n /) /1 /= /2/ /(/( max/1 / k / n j /k j /) /2/+ /:/:/: /+/(m a x/1 / k / n j /k j /) /2/) /1 /= /2/=/=/( n /( max/1 / k / n j /k j /) /2/) /1 /= /2/= p n k x k/1 /;;k x k/1 /= max/1 / k / n j /k j/j //1 j /+ /:/:/: /+ j /n j/ n max/1 / k / n j /k j /= n k x k/1 /: /2Prop osition /1/./5/./4/. A space with scalar pro duct X is a normed spacewith the normk x k /= q h x /;; x i /:Pr o of/. Let us v erify the v alidit y of conditions /1/-/3/:k x k /=/0 /, q h x /;; x i /=/0 /,h x /;; x i /=/0 /, x /= /0 /;;k / x k /= q h / x /;;/ x i /= q j / j /2h x /;; x i /= j / j q h x /;; x i /= j / jk x k /;;k x /+ y k /= q h x /+ y /;; x /+ y i /= q h x /;; x i /+ h x /;; y i /+ h y /;; x i /+ h y /;; y i /=/= q k x k /2/+ h x /;; y i /+ h x /;; y i /+ k y k /2/= q k x k /2/+/2 /<h x /;; y i /+ k y k /2// q k x k /2/+/2 j /( x /;; y /) j /+/+ k y k /2/ q k x k /2/+/2 k x kk y k /+ k y k /2// q /( k x k /+ k y k /) /2/= k x k /+ k y k /:/< /( x /;; y /)i st h e notation for the real part of the complex n um be r h x /;; y i /:/1/5 Prop osition /1/./5/./5/. In the normed space with scalar pro duct the p ar al/-lelo gr am rule /:k x /+ y k /2/+ k x /; y k /2/=/2 /( k x k /2/+ k y k /2/) /:holds/.Pr o of/. By the immediate c hec k/, w e getk x /+ y k /2/+ k x /; y k /2/= h x /+ y /;; x /+ y i /+ h x /; y /;; x /; y i /=/= h x /;; x i /+ h x /;; y i /+ h y /;; x i /+ h y /;; y i /+ h x /;; x i/;h x /;; y i/;h y /;; x i /+ h y /;; y i /=/=/2 /( k x k /2/+ k y k /2/) /:De/nition /1/./5/./3/. It is said that the sequence f x /( k /)g of the elemen ts ofthe space C ncon v erges with resp ect to the p /-norm to the elemen t x /2 C niflimk /!/1 / / / x /( k /)/; x / / / p /=/0 /:In this case w e shall write x /( k /)/! x /:Remark /1/./5/./1/. Since all the p /-norms of the space C nare equiv alen t/,this implies that the con v ergence of the sequence f x /( k /)g with resp ect to the/ /-norm will yield its con v ergence with resp ect to the / /-norm/.Problem /1/./5/./2/. Sho w that if x /2 C n/, then limp /!/1 k x kp /= k x k/1 /:Problem /1/./5/./3/. Sho w that if x /2 C n/, thenk x kp / c /( k/< x kp /+ k/= x kp /) /;;where x /= /< x /+ i /= x and /< x /, /= x /2 R n/. Find suc h a constan t cn /,t h a tcn /( k /< x k/2 /+ k/= x k/2 /) /k x k/2 /8 x /2 C n/:De/nition /1/./5/./4/. A v ector bx /2 R nis called an appr oximation to thev ector x /2 R nif it di/ers little from x in some sense/.De/nition /1/./5/./5/. In case of the /xed norm k/
k /,t h e quan tit y/"abs /= jj bx /; x jjis called the absolute err or of the appro ximation bx to the v ector x /;; and thequan tit y/"re l /= k bx /; x k /= k x k/1/6 is called the r elative err or of the appro ximation /( x /6/= /0 /)/.In case of the /1/; norm the relativ e error can b e considered as an indexof the correct signi/can t digits/. Namely /, if k bx /; x k/1 /= k x k/1 / /1/0 /; k/;; thenthe greatest comp onen t of the v ector bx has k correct signi/can t digits/.Example /1/./5/./2/. Let x /=/[/2 /: /5/4/3/;; /0 /: /0/6/3/5/6/] Tand bx /=/[/2 /: /5/4/1/;; /0 /: /0/6/9/3/7/] T/:Find /"abs and /"re l /, and then the n um be r of the correct signi/can t digitsof the greatest comp onen t of the appro ximation bx b y /"re l /. W e get bx /;x /=/[ /; /0 /: /0/0/2/;; /0 /: /0/0/5/8/1/] T/;; /"abs /= k bx /; x k/1 /= /0 /: /0/0/5/8/1 and k x k/1 /= /2 /: /5/4/3 and/"re l / /0 /: /0/0/2/3 / /1/0 /; /3/) k /= /3 /: Th us the greatest comp onen t b//1 of bx hasthree correct signi/can t digits/. A t the same time/, the comp onen t b//2 has onlyone correct signi/can t digit/./1/./1/./6 Orthogonal V ectorsDe/nition /1/./6/./1/. The v ectors x and y of the v ector space with scalarpro duct X are called ortho gonal if h x /;; y i /=/0 /: W e write x /? y to indicate theorthogonalit yo f v ectors x and y /: A v ector x of the v ector space X is calledortho gonal to the set Y / X if x /? y /8 y /2 Y/:Problem /1/./6/./1/. /Find all v ectors that are orthogonal b oth to the v ectora /= h/4 /0 /6 /; /2 /0 iTand b /= h/2 /1 /; /1 /1 /1 iT/:De/nition /1/./6/./2/. The sets Y and Z of the v ector space X are calledortho gonal if y /? z /8 y /2 Y and /8 z /2 Z/:De/nition /1/./6/./3/. A sequence f x /( k /)g of v ectors of the v ector space withscalar pro duct X is called a Cauchy se quenc e if for an y / /> /0 there is anatural n um be r n/0 suc h that for all m /2 N and n/>n/0jj x /( n /)/; x /( n /+ m /)jj /= q h x /( n /)/; x /( n /+ m /)/;; x /( n /)/; x /( n /+ m /)i /</" /:De/nition /1/./6/./4/. A v ector space with scalar pro duct X is called c om/-plete if ev ery Cauc h y sequence is con v ergen t to a p oin to f t h e space X /.De/nition /1/./6/./5/. Av ector space with complex scalar pro duct is called aHilb ert sp ac e H if it turns out to b e complete with resp ect to the con v ergenceb y the norm k x k /= q h x /;; x i /./1/7 Prop osition /1/./6/./1/. The space C nwith the scalar pro duct h x /;; y i /= Pnk /=/1 /k /kis a Hilb ert space/.Prop osition /1/./6/./2/. The space L/2 /[ //;; / /] of all square/-in tegrable functionson the in terv al /[ //;; / /] with the scalar pro duct h x /;; y i /= R// x /( t /) y /( t /) dt is a Hilb ertspace/.Prop osition /1/./6/./3/. Orthogonalit y of v ectors in the v ector space withscalar pro duct X has the follo wing prop erties /(/1/-/4/)/:/1/. x /? x /, x /= /0 /;;/2/. x /? y /, y /? x /;;/3/. x /?f y/1 /;;/:/:/: /;; yk g/) x /? /( y/1 /+ /:/:/: /+ yk /)/;;/4/. x /? y /) x /? / y /8 / /2 K /;;orthogonalit yo f v ectors in a Hilb ert space has an additional prop ert y/:/5/. x /? yn /( n /= /1 /;;/2 /;;/3 /;; /:/:/: /) /^ yn /! y /) x /? y /:Let us pr ove these assertions/:x /? x /,h x /;; x i /=/0 /, x /= /0 /;;x /? y /,h x /;; y i /=/0 /, h y /;; x i /=/0 /,h y /;; x i /=/0 /, y /? x /;;x /?f y/1 /;;/:/:/: /;; yk g/, x /? y/1 /^ /:/:/: /^ x /? yk /,h x /;; y/1 i /=/0 /^ /:/:/: /^h x /;; yk i /=/0/)/)h x /;; y/1 i /+ /:/:/: /+ h x /;; yk i /=/0 /,h x /;; y/1 /+ /:/:/: /+ yk i /=/0 /, x /? /( y/1 /+ /:/:/: /+ yk /)/;;x /? y /,h x /;; y i /=/0 /, / h x /;; y i /=/0 /8 / /2 K /,/,h x /;;/ y i /=/0 /8 / /2 K /, x /? / y /;;x /? yn /8 n /2 N /^ yn /! y /,h x /;; yn i /=/0 /^ k yn /; y k/! /0 /)/)h x /;; yn i /=/0 /^j h x /;; yn i/; h x /;; y i j /= jh x /;; yn /; y ij / k x kk yn /; y k/! /0 /)/)h x /;; y i /= /0 /, x /? y /:De/nition /1/./6/./6/. The ortho gonal c omplement of the set Y / X is theset Y /?of all v ectors of the space X that are orthogonal to the set Y /, i/.e/./,Y /?/= f x /: /( x /2 X /) /^ /( x /? y /8 y /2 Y /) g /:Problem /1/./6/./2/. /Let U /= span /h/1 /0 /1 iT/;; h/0 /2 /1 iT // R /3/:Find the orthogonal complemen to ft h e set U/:/1/8 Prop osition /1/./6/./4/. If X is a v ector space with scalar pro duct/, x /2 X /;;Y / X and x /? Y/;; then x /? span Y /: If/, in addition/, X is complete/, i/.e/./, is aHilb ert space/, then x /? span Y /:Pr o of/. By assertions /3 and /4 of prop osition /1/./6/./3/, x /? span Y /. If y /2 span Y /;;i/.e/./, /9 yn /2 span Y suc h that yn /! y/;; then/, due to the orthogonalit y x /? ynand assertion /5 of prop osition /1/./6/./3/, w e get x /? y /, i/.e/./, x /? spanY /:Prop osition /1/./6/./5/. The orthogonal complemen t Y /?of the set Y / X isa subspace of the space X /: The orthogonal complemen t Y /?of the set Y / His a closed subspace of the Hilb ert space H /;; i/.e/./, Y /?is a subspace of thespace H that con tains all its b oundary p oin ts/.Pr o of/. Due to the prop osition /1/./2/./1/, it is su/cien t for the pro of of the/rst assertion of prop osition /1/./6/./5 to sho w that Y /?is closed with resp ect tov ector addition and scalar m ultiplication/. It will follo w from assertion /5 ofthe same prop osition/, it holds the second assertion of prop osition /1/./6/./5 to o/.Prop osition /1/./6/./6/. If Y is a closed subspace of the Hilb ert spaceH /;; then eac h x /2 H can be expressed uniquely as the sum x /= y /+ z /;; /,y /2 Y /;; z /2 Y /?/:Corollary /1/./6/./1/. If Y is a closed subspace of the Hilb ert space/, thenthe space H can be presen ted as the direct sum H /= L / L /?of the closedsubspaces L and L /?/,a n d /( L /?/) /?/= L /:De/nition /1/./6/./7/. The distanc e of the v ector x of the Hilb ert space Hfrom the subsp ac e Y / H is de/ned b y the form ula/ /( x /;; Y /)/= infy /2 Y k x /; y k /:Prop osition /1/./6/./7/. If Y is a closed subspace of the Hilb ert space Hand x /2 H /, then there exists a uniquely de/ned y /2 Y suc h that k x /; y k /=/ /( x /;; Y /) /:De/nition /1/./6/./8/. The v ector y in prop osition /1/./6/./7 is called the ortho g/-onal pr oje ction of x on to the subspace Y/.De/nition /1/./6/./9/. A ve ctor system S /= f x/1 /;;/:/:/: /;; xk g is called ortho gonalif /( xi /;; xj /) /= k xi k /2/ij /;; where /ij is the Kronec k er delta/. The v ector systemS /= f x/1 /;;/:/:/: /;; xk g is called orthonormal if /( xi /;; xj /)/= /ij /.Example /1/./6/./1/. The v ector system f ek g /( k /= /1 /: n/)/, where ek /=/[/0/;; /0/;; /:/:/: /;;/0k /; /1 ze r o s /;;/1 /;; /0 /;; /:/:/: /;;/0n /; k z er os /] T/;; is orthonormal in C n/./1/9 Example /1/./6/./2/. The v ector systemf /1 /= p /2 //;; /(cos t /) /= p //;; /(sin t /) /= p //;; /(cos /2 t /) /= p //;; /(sin /2 t /) /= p //;; /:/:/: gis orthonormal in L/2 /[ /; //;; / /] /:Example /1/./6/./3/. The v ector system f exp /( i /2 /k t /) gk /2 Z is orthonormal inL/2 /[/0/;; /1/] /: T ruely /,/( xk /;; xj /)/= Z/1/0 exp /( i /2 /k t /) exp /( i /2 /j t /) dt /= Z/1/0 exp /( i /2 / /( k /; j /) t /) dt /=/= /(/(exp /( i /2 / /( k /; j /)/) /; /1/) /= /( i /2 / /( k /; j /)/) /=/0 /, kui k /6/= j /;;/1 /;; as k /= j/:Prop osition /1/./6/./8/. /( Gram/-Sc hmidt orthogonalization theorem/)/. Iff x/1 /;;/:/:/: /;; xk g is a linearly indep enden tv ector system in the v ector space withscalar pro duct H /, then there exists an orthonormal system f /"/1 /;;/:/:/: /;;/"k g suc hthat span f x/1 /;;/:/:/: /;; xk g /= span f /"/1 /;;/:/:/: /;;/"k g /:Let us pr ove this assertion b y complete induction/. In the case k /= /1/,w e de/ne /"/1 /= x/1 /= k x/1 k /;; and/, ob viously /, span f x/1 g /= span f /"/1 g /: So w e ha v esho wn the existence of the induction base/. W e ha v e to sho w the admiss/-abily of the induction step/. Let us assume that the prop osition holds fork /= i /; /1/, i/.e/./, there exists an orthonormal system f /"/1 /;;/:/:/: /;;/"i /; /1 g suc h thatspan f x/1 /;;/:/:/: /;; xi /; /1 g /= span f /"/1 /;;/:/:/: /;;/"i /; /1 g /: No w w e consider the v ectoryi /= //1 /"/1 /+ /:/:/: /+ /i /; /1 /"i /; /1 /+ xi /;; /j /2 K /:Let us c ho ose the co e/cien ts // /( / /=/1/: i/-/1/) so that yi /? /"/ /( / /=/1/: i/-/1/) /;; i/.e/,/( yi /;;/"/ /)/= /0 /: W e get i /; /1 conditions/:// /( /"/ /;;/"/ /)/+/( xi /;;/"/ /)/= /0 /;; ehk // /= /; /( xi /;;/"/ /) /( / /=/1/: i/-/1/) /:Th us/,yi /= xi /; /( xi /;;/"/1 /) /"/1 /; /:/:/: /; /( xi /;;/"i /; /1 /) /"i /; /1 /:No w w e c hose /"i /= yi /= k yi k /: Since/"/ /2 span f x/1 /;;/:/:/: /;; xi /; /1 g /( / /=/1/: i/-/1 /) /;;w e get/, b y the construction of v ectors yi and /"i /, /"i /2 span f x/1 /;;/:/:/: /;; xi g /:Hencespan f /"/1 /;;/:/:/: /;;/"i g/ span f x/1 /;;/:/:/: /;; xi g /:/2/0 F rom the represen tation of the v ector yi w e see that xi is a linear com binationof v ectors /"/1 /;;/:/:/: /;;/"i /:Th us/,span f x/1 /;;/:/:/: /;; xi g/ span f /"/1 /;;/:/:/: /;;/"i g /:Finally /,span f x/1 /;;/:/:/: /;; xi g /= span f /"/1 /;;/:/:/: /;;/"i g /:Example /1/./6/./4/. Giv en av ector system f x/1 /;; x/2 /;; x/3 g in R /4/, wherex/1 /= /[/1/;; /0/;; /1/;; /0/] T/;; x/2 /= /[/1/;; /1/;; /1/;; /0/] T/;; x/3 /= /[/0/;; /1/;; /0/;; /1/] T/:Find suc h an orthogonal system f /"/1 /;;/"/2 /;;/"/3 g /, for whic hspan f x/1 /;; x/2 /;; x/3 g /= span f /"/1 /;;/"/2 /;;/"/3 g /:T o apply the orthogonalization pro cess of prop osition /1/./6/./8/, w ec hec k /rst thesystem f x/1 /;; x/2 /;; x/3 g for the linearly indep endence /(one can omit this pro cess/,to o/, b ecause the situation will b e clear in the course of the orthogonalization/:/2/6/4 /1 /0 /1 /0/1 /1 /1 /0/0 /1 /0 /1 /3/7/5 I I/-I/ /2/6/4 /1 /0 /1 /0/0 /1 /0 /0/0 /1 /0 /1 /3/7/5 I I I/-I I/ /2/6/4 /1 /0 /1 /0/0 /1 /0 /0/0 /0 /0 /1 /3/7/5 /)the system f x/1 /;; x/2 /;; x/3 g is linearly indep enden t/. No w w e /nd/"/1 /= x/1 /= k x/1 k /=/[ /1 /= p /2/;; /0/;; /1 /= p /2/;; /0/] T/:F or y/2 w e get/:y/2 /= x/2 /; /( x/2 /;;/"/1 /) /"/1 /= /[/1/;; /1/;; /1/;; /0/] T/; p /2/[/1 /= p /2 /;;/0 /;;/1 /= p /2/;; /0/] T/= /[/0/;; /1/;; /0/;; /0/] T/:As k y/2 k /=/1 /;;/"/2 /= y/2 /= k y/2 k /= /[/0/;; /1/;; /0/;; /0/] T/: The v ector y/3 can b e expressed inthe form/:y/3 /= x/3 /; /( x/3 /;;/"/1 /) /"/1 /; /( x/3 /;;/"/2 /) /"/2 /=/= /[/0/;; /1/;; /0/;; /1/] T/; /0 /
/[/1 /= p /2/;; /0/;; /1 /= p /2 /;;/0 /] T/; /1 /
/[ /0 /;;/1 /;;/0 /;;/0 /] T/= /[/0/;; /0/;; /0/;; /1/] T/:Th us/,/"/3 /= y/3 /= k y/3 k /= /[/0/;; /0/;; /0/;; /1/] T/:/2/1 Example /1/./6/./5/. Giv en a linearly indep enden tv ector system f x/1 /;; x/2 /;; x/3 gin L/2 /[ /; /1/;; /1/]/, where x/1 /= /1 /;; x/2 /= t and x/3 /= t /2/: Find an orthogonal systemf /"/1 /;;/"/2 /;;/"/3 g /, suc h thatspan f x/1 /;; x/2 /;; x/3 g /= span f /"/1 /;;/"/2 /;;/"/3 g /:Chec k that the system f x/1 /;; x/2 /;; x/3 g is linearly indep enden t/. The /rst v ectoris/"/1 /= x/1 /= k x/1 k /=/1 /= p /2 /:The v ector y/2 can be expressed in the form/:y/2 /= x/2 /; /( x/2 /;;/"/1 /) /"/1 /= t /; /( Z/1/; /1 t /
/( /1 p /2 /) dt /) t /= t /; /0 /
t /= t/:Th us/,/"/2 /= y/2 /= k y/2 k /= t/= s Z/1/; /1 t /
tdt /= t/= s /2 /3 /= s /3 /2 t/:The v ector y/3 can be expressed in the form/:y/3 /= x/3 /; /( x/3 /;;/"/1 /) /"/1 /; /( x/3 /;;/"/2 /) /"/2 /=/= t /2/; /( Z/1/; /1 t /2/
/( /1 p /2 /) dt /) /1 p /2 /; /( Z/1/; /1 t /2 /( s /3 /2 t /) dt /) s /3 /2 t /=/= t /2/; /1 /2 /
/2 /3 /; /0/= t /2/; /1 /3 /:Therefore/,/"/3 /= y/3 /= k y/3 k /=/( t /2/; /1 /3 /) /= s Z/1/; /1 /( t /2/; /1 /3 /) /( t /2/; /1 /3 /) dt /=/=/( t /2/; /1 /3 /) /= s /2 /5 /; /4 /9 /+ /2 /9 /= s /4/5 /8 /( t /2/; /1 /3 /)/= /3 /2 s /5 /2 /( t /2/; /1 /3 /) /:The functions /"/1 /;;/"/2 and /"/3 are the normed Legendre p olynomials on /[ /; /1/;; /1/] /:Problem /1/./6/./3/. Sho w that a v ector system f x/1 /;;/:/:/: /;;xn g with pairwiseorthogonal elemen ts is linearly indep enden t/./2/2 /1/./2 Matrices/1/./2/./1 Notation for a Matrix and Op erations with MatricesThe v ector space of all m / n /; matrices with real elemen ts will b e denotedb y R m / nandA /2 R m / n/, A /=/( aik /)/= /2/6/6/4 a/1/1 /
/
/
a/1 n/././. /././.am /1 /
/
/
amn /3/7/7/5 /;; aik /2 R /:The elemen t of the matrix A that stands in the i /; th ro w and k /; th columnwill b e denoted b y aik or A /( i/;; k /)o r /[ A /]ik /: The main op erations with matricesare follo wing/:/ transp osition of matrices /( R m / n/! R n / m/)B /= A T/, bik /= a/;;/ addition of matrices /( R m / n/ R m / n/! R m / n/)C /= A /+ B /, cik /= aik /+ bik /;;/ m ultiplication of matrices b y an um be r /( R / R m / n/! R m / n/)B /= /A /, bik /= /aik/ m ultiplication of matrices /( R m / p/ R p / n/! R m / n/)C /= AB /, cik /= pXj /=/1 aij bjk /:Problem /2/./1/./1/. /LetA /= /"a c eb d f /#/;; B /= /2/6/4 k nl pm r /3/7/5 /:Find the matrix AB /:/2/3 Problem /2/./1/./2/. /LetA /= /2/6/6 /6 /6 /6/6/6 /6 /6 /6 /6/6/4 /1 /0 /0 /
/
/
/0 /0/1 /1 /0 /./././0 /0/0 /1 /1 /./././0 /0/./. /. /././. /././. /././. /././. /./././0 /0 /././. /./././1 /0/0 /0 /0 /
/
/
/1 /1 /3/7/7 /7 /7 /7/7/7 /7 /7 /7 /7/7/5 /2 R n / n/:Find the matrix A n /; /1/:Problem /2/./1/./3/. /LetA /= /"/1 /1/1 /1 /#/:Pro v e thatA n/=/2 n /; /1A /( n /2 N /) /:Example /2/./1/./1/. /Let us sho w that m ultiplication of matrices is notcomm utativ e/. LetA /= /"/1 /4/3 /2 /#/;; B /= /"/; /2 /5/1 /2 /#/:W e /nd the pro ducts/:AB /= /"/1 /4/3 /2 /#/"/; /2 /5/1 /2 /#/= /"/2 /1/3/; /4 /1/9 /#/;;BA /= /"/; /2 /5/1 /2 /#/"/1 /4/3 /2 /#/= /"/1/3 /2/7 /8 /#/:As AB /6/= BA do es not hold for the example/, m ultiplication of matrices isnot comm utativ e in general/.Prop osition /2/./1/./1/. If A /2 R m / pand B /2 R p / n/;; then/( AB /) T/= B TA T/:Pr o of/. If C /=/( AB /) T/;; thencik /=/[ /( AB /) T/]ik /=/[ AB /]ki /= pXj /=/1 akj bji /:/2/4 If D /= B TA T/;; w e also ha v edik /=/[ B TA T/]ik /= pXj /=/1 /[ B T/]ij /[ A T/]jk /= pXj /=/1 /[ B /]ji /[ A /]kj /=/= pXj /=/1 akj bji /= cik /: /2De/nition /2/./1/./1/. A matrix A /2 R n / nis called symmetric if A T/= Aand skew/-symmetric if A T/= /; A/:Problem /2/./1/./4/. /Is matrix A symmetric or sk ew/-symmetric ifa /) A /= /2/6/4 /; /1 /3 /2/3 /1 /3/2 /3 /; /1 /3/7/5 /;; b /) A /= /2/6/4 /0 /2 /; /4/; /2 /1 /; /7/4 /7 /2 /3/7/5 /;; c /) A /= /2/6/4 /2 /; /3 /5/3 /1 /2/; /5 /1 /4 /3/7/5 /:Prop osition /2/./1/./2/. Eac h matrix A /2 R n / ncan be expressed as a sumof a symmetric matrix and as k ew/-symmetric matrix/.Pr o of/. Eac h matrix A /2 R n / ncan be expressed as A /= B /+ C/;; whereB /=/( A /+ A T/) /= /2a n d C /=/( A /; A T/) /= /2 /: AsB T/=/( /( A /+ A T/) /= /2/) T/=/( A T/+ A /) /= /2/= BandC T/=/( /( A /; A T/) /= /2/) T/= C /=/( A T/; A /) /= /2/= /; C/;;the prop osition holds/. /2Problem /2/./1/./5/. /Represen t the matrixA /= /2/6/6 /6/4 /2 /; /3 /5 /1/; /3 /; /2 /3 /0/3 /; /7 /0 /6/4 /5 /2 /4 /3/7/7 /7/5as a sum of a symmetric and as k ew/-symmetric matrix/.De/nition /2/./1/./2/. If A is a m / n /; matrix with complex elemen ts/, i/.e/./,A /2 C m / n/;; then the tr ansp ose d skew/-symmetric matrix A Hwill be de/nedb y the equalit yB /= A H/, bik /= aki /:/2/5 De/nition /2/./1/./3/. A matrix A /2 C n / nis called an Hermitian matrix ifA H/= A/:Problem /2/./1/./6/. /Is matrix A an Hermitian matrix ifa /) A /= /2/6/4 i /; /2/+ i /; /5/+ /3 i/2/+ i /5 i /; /2/+ i/5/+/3 i /2/+ i /; /8 i /3/7/5 /;; b /) A /= /2/6/4 /5 /2/+/3 i /1/+ i/2 /; /3 i /; /3 /; /2 i/1 /; i /2 i /0 /3/7/5 /:Problem /2/./1/./7/. /Let A /2 C m / n/: Sho w that matrices AA Hand A HAare Hermitian matrices/.The matrix A /2 C m / ncan be expressed b oth b y the column/-v ectorsck /(k /= /1/: n/) of the matrix A and b y the ro w/-v ectors r Ti /( i/= /1 /: m /) ofthe transp ose of matrix A /(/\pasting/" the matrices of the column/-v ectors orof the transp osed ro w/-v ectors/)A /= hc/1 /
/
/
cn i/ hc/1 /;; /
/
/
/;; cn i/= /2/6/6/4 r T/1/././.r Tm /3/7/7/5 /;;where ck /2 C mand ri /2 C nandri /= /2/6/6/4 ai /1/././.ain /3/7/7/5 /;; ck /= /2/6/6/4 a/1 k/././.amk /3/7/7/5 /:Example /2/./1/./2/. Let us demonstrate these notions on a matrix A /2 R /3 / /2/:A /= /2/6/4 /2 /3/4 /1/3 /2 /3/7/5 /) c/1 /= /2/6/4 /2/4 /3 /3/7/5 /^ c/2 /= /2/6/4 /3/1 /2 /3/7/5 /^r/1 /= /"/2/3 /#/^ r/2 /= /"/4/1 /#/^ r/3 /= /"/3/2 /#/^r T/1 /= h/2 /3 i/^ r T/2 /= h/4 /1 i/^ r/3 /= h/3 /2 i/^A /= hc/1 c/2 i/= hc/1 /;; c/2 i/= /2/6/4 r T/1r T/2r T/3 /3/7/5 /:/2/6 If A /2 R m / n/;; then A /( i/;; /:/) denotes the i /; th ro w of the matrix A /, i/.e/./,A /( i/;; /:/) /= hai /1 /
/
/
ain i/;;and A /(/: /;; k /) denotes the k /-th column of the matrix A /, i/.e/./,A /(/: /;; k /)/= /2/6/6/4 a/1 k/./. /.amk /3/7/7/5 /:If /1 / p / q/< n /^ /1 / r / m/;; thenA /( r /;; p /: q /)/= harp /
/
/
arq i/2 R /1 / /( q /; p /+/1/)and if /1 / p / n /^ /1 / r / s / m/;; thenA /( r /: s/;; p /)/= /2/6/6/4 arp/././.asp /3/7/7/5 /2 R s /; r /+/1/:If A /2 R m / nand i /=/( i/1 /;;/:/:/: /;;ip /) and k /=/( k/1 /;;/:/:/: /;;kq /) /;; wherei/1 /;;/:/:/: /;;ip /2f /1/;; /2/;; /:/:/: /;; m g /^ k/1 /;;/:/:/: /;;kq /2f /1/;; /2/;; /:/:/: /;; n g /;;then the corresp onding submatrix isA /( i /;; k /)/= /2/6/6/4 A /( i/1 /;;k/1 /) /
/
/
A /( i/1 /;;kq /)/././. /././.A /( ip /;;k/1 /) /
/
/
A /( ip /;;kq /) /3/7/7/5 /:Example /2/./1/./3/. IfA /= /2/6/6/6/4 /1 /4 /; /1 /2 /; /4 /8/2 /; /2 /4 /1 /3 /5/5 /6 /; /7 /2 /; /1 /9/4 /5 /6 /; /4 /9 /1 /3/7/7/7/5and i /= /(/2/;; /4/) and k /= /(/1/;; /3/;; /5/) /;; thenA /( i /;; k /)/= /"/2 /4 /3/4 /6 /9 /#/:/2/7 /1/./2/./2 Band Matrices and Blo c k MatricesDe/nition /2/./2/./1/. A matrix whose elemen ts di/eren t from zero are onlyon the main and some adjacen t diagonals is called a b and matrix /.De/nition /2/./2/./2/. It is said that the matrix A /2 R m / nis a b and matrixwith the lower b andwidth p if/( i/> k /+ p /) /) aik /=/0and with the upp er b andwidth q if/( k/> i /+ q /) /) aik /=/0 /;;and with the bandwidth p /+ q /+/1 /:Example /2/./2/./1/. The matrixA /= /2/6/6/6 /6 /6 /6/6/6/4 / / /0 /0 /0 /0 /0/ / / /0 /0 /0 /0/ / / / /0 /0 /0/0 / / / / /0 /0/0 /0 / / / / /0/0 /0 /0 / / / / /3/7/7/7 /7 /7 /7/7/7/5is a band matrix b ecause all the elemen ts di/eren t from zero are on the mainand t w o lo w er and one upp er diagonals/. The lo w er bandwidth of the matrixA is /2 b ecause aik /= /0 as i />k /+/2 /;; and the upp er bandwidth is /1 b ecauseaik /= /0 as k /> i /+/1 /: The bandwidth of the matrix is /2/+ /1/+/1 /= /4 /: Theelemen ts of the matrix that are necessarily not zeros are denoted b y crosses/.Some of the most imp ortan t t yp es of band matrices are presen ted intable /2/./2/./1/. If D /2 R m / nis a diagonal matrix/, q /= min f m/;; n g and di /= dii /;;then the notation D /= diag /( d/1 /;;/:/:/: /;;dq /) /: will b e used/./2/8 T able /2/./2/./1/. The matrice/'s t yp e Lo w er bandwidth Upp er bandwidth diagonal matrix /0 /0 upp er triangular matrix /0 n/-/1 lo w er triangular matrix m/-/1 /0 tridiagonal matrix /1 /1 upp er tridiagonal matrix /0 /1 lo w er tridiagonal matrix /1 /0 upp er Hessen b erg matrix /1 n/-/1 lo w er Hessen b erg matrix m/-/1 /1 Problem /2/./2/./1/. /Find the t yp e/, lo w er bandwidth/, upp er bandwidth andbandwidth of the matrix A ifA /= /2/6/6/6/6 /6 /6/4 /1 /3 /0 /0 /0/4 /2 /1 /1 /0/0 /2 /3 /4 /1/0 /0 /5 /4 /6/0 /0 /0 /6 /5 /3/7/7/7/7 /7 /7/5 /;; A /= /2/6/6/6 /6 /6 /6/6/6 /6 /6 /6/6/6 /6 /6/4 /1 /1 /0 /0 /
/
/
/0 /1/2 /2 /1 /0 /./././0 /0/1 /2 /3 /1 /./././0 /0/0 /1 /2 /4 /././. /./././0/./. /. /././. /././. /././. /./././
/
/
/./. /./0 /0 /0 /./././
/
/
n /; /1 /1/0 /0 /0 /0 /
/
/
/2 n /3/7/7/7 /7 /7 /7/7/7 /7 /7 /7/7/7 /7 /7/5 /:De/nition /2/./2/./3/. A matrix A /=/( A// /) /2 R m / nis called a q / r /; blo ckmatrix ifA /= /2/6/6/4 A/1/;;/1 /:/:/: A/1/;; r/././. /././.Aq /;;/1 /:/:/: Aq /;; r /3/7/7/5 m/1mq /;;n/1 nrwhere m/1 /+ /:/:/: /+ mq /= m and n/1 /+ /:/:/: /+ nr /= n and A// is a m/ / n/ /; matrix/.Example /2/./2/./2/. The matrixA /= /2/6/6 /6/4 a a a b ba a a b ba a a b bc c c d d /3/7/7 /7/5/2/9 is a /2 / /2 /; blo c k matrix/, where m/1 /=/3 /;; m/2 /=/1 /;; n/1 /=/3 and n/2 /=/2 andA/1/;;/1 /= /2/6/4 a a aa a aa a a /3/7/5 /;; A/1/;;/2 /= /2/6/4 b bb bb b /3/7/5 /;; A/2/;;/1 /= hc c c i/;; A/2/;;/2 /= hd d i/:LetB /= /2/6/6/4 B/1/;;/1 /:/:/: B/1/;; r/./. /. /./. /.Bq /;;/1 /:/:/: Bq /;; r /3/7/7/5 m/1mq /;;n/1 nrand C /= A /+ B/: ThenC /= /2/6/6/4 C/1/;;/1 /:/:/: C/1/;; r/././. /././.Cq /;;/1 /:/:/: Cq /;; r /3/7/7/5 /= /2/6/6/4 A/1/;;/1 /+ B/1/;;/1 /:/:/: A/1/;; r /+ B/1/;; r/././. /././.Aq /;;/1 /+ Bq /;;/1 /:/:/: Bq /;; r /+ Bq /;; r /3/7/7/5 /:Prop osition /2/./2/./1/. If A /2 R m / p/;; B /2 R p / nand C /= AB are blo c kmatrices/:A /= /2/6/6/6 /6 /6 /6/6/4 A/1/;;/1 /:/:/: A/1/;; r/././. /././.A/ /;;/1 /:/:/: A/ /;; r/./. /. /./. /.Aq /;;/1 /:/:/: Aq /;; r /3/7/7/7 /7 /7 /7/7/5 m/1m/mq /;;p/1 prB /= /2/6/6/4 B/1/;; /1 /:/:/: B/1/;; / /:/:/: B/1/;; s/./. /. /./. /. /./. /.Br /;;/1 /:/:/: Br /;; / /:/:/: Br /;; s /3/7/7/5 p/1pr /;;n/1 n/ nsC /= /2/6/6/6/6 /6 /6 /6/4 C/1/;;/1 /:/:/: C/1/;; / /:/:/: C/1/;; s/././. /././. /././.C/ /;;/1 /:/:/: C/ /;; / /:/:/: C/ /;; s/./. /. /./. /. /./. /.Cq /;;/1 /:/:/: Cq /;; / /:/:/: Cq /;; s /3/7/7/7/7 /7 /7 /7/5 m/1m/mq /;;n/1 n/ ns/3/0 where /1 / / / q/;; /1 / / / s/;; m/1 /+ /:/:/: /+ mq /= m/;; p/1 /+ /:/:/: /+ ps /= p/;;n/1 /+ /:/:/: /+ nr /= n /, thenC/ /;; / /= rX/ /=/1 A/ /;; / B/ /;; / /( / /= /1/: q /^ / /=/1 /: s/)/.Pr o of/. Let/ /= m/1 /+ /:/:/: /+ m/ /; /1 /;; / /= n/1 /+ /:/:/: /+ n/ /; /1 /;; /1 / / / r /;;/ /= p/1 /+ /:/:/: /+ p/ /; /1 /;; m/0 /= n/0 /= p/0 /=/0 /:As /[ C/ /;; / /]i /;; k is an elemen t of the blo c k C/ /;; / of the matrix C standing in thei /; th ro w and k /; th column of this blo c k/, and /[ A/ /;; / /]i /;; j is an elemen t of theblo c k A/ /;; / of the matrix A standing in the i /; th ro w and j /; th column of thisblo c k/, and /[ B/ /;; / /] is an elemen t of the blo c k B/ /;; / of the matrix B standingin the j /; th ro w and k /; th column/, then/[ C/ /;; / /]i /;; k /= c/ /+ i /;; / /+ k /;; /[ A/ /;; / /]i /;; j /= a/ /+ i /;; / /+ j /;; /[ B/ /;; / /]j /;; k /= b/ /+ j /;; / /+ k /:Therefore/,/[ C/ /;; / /]i /;; k /= c/ /+ i /;; / /+ k /= pXj /=/1 a/ /+ i /;; j bj /;; / /+ k /=/= p/1Xj /=/1 a/ /+ i /;; j bj /;; / /+ k /+ p/1 /+ p/2Xj /= p/1 /+/1 a/ /+ i /;; j bj /;; / /+ k /+ /:/:/: /+ pXj /= p/1 /+ p/2 /+ /:/:/: /+ pr /; /1 /+/1 a/ /+ i /;; j bj /;; / /+ k /=/= p/1Xj /=/1 /[ A/ /;;/1 /]i /;; j /[ B/1/;; / /]j /;; k /+ p/2Xj /=/1 /[ A/ /;;/2 /]i /;; j /[ B/2/;; / /]j /;; k /+ /:/:/: /+ prXj /=/1 /[ A/ /;; r /]i /;; j /[ Br /;; / /]j /;; k /=/=/[ A/ /;;/1 B/1/;; / /]i /;; k /+/[ A/ /;;/2 B/2/;; / /]i /;; k /+ /:/:/: /+/[ A/ /;; r Br /;; / /]i /;; k /=/[ rXj /=/1 A/ /;; j Bj /;; / /]i /;; k /:Therefore/, all the corresp onding elemen ts of the matrices C/ /;; / and Ps/ /=/1 A/ /;; / B/ /;; /are equal/, and our prop osition holds/. /2Corollary /2/./2/./1/. If A /2 R m / p/;; B /2 R p / n/;; A /= /2/6/6/4 A/1/./. /.Aq /3/7/7/5 m/1mq /;;B /= hB/1 /:/:/: Br i/;;/3/1 n/1 nrand m/1 /+ /:/:/: /+ mq /= m and n/1 /+ /:/:/: /+ nr /= n/;; thenAB /= C /= /2/6/6/4 C/1/;;/1 /:/:/: C/1/;; r/././. /././.Cq /;;/1 /:/:/: Cq /;; r /3/7/7/5 m/1mq /;;n/1 nrwhere C// /= A/ B/ /( / /=/1/: q /^ / /= /1/: r/) /.Corollary /2/./2/./2/. If A /2 R m / p/;; B /2 R p / n/;;A /= hA/1 /:/:/: As i/;;p/1 psB /= /2/6/6/4 B/1/./. /.Bs /3/7/7/5 p/1psand p/1 /+ /:/:/: /+ ps /= p/;; then AB /= C /= Ppk /=/1 Ak Bk /:Example /2/./2/./3/. It holds/"A/1/;; /1 A/1/;; /2A/2/;; /1 A/2/;; /2 /#/"x/1x/2 /#/= /"A/1/;; /1 x/1 /+ A/1/;; /2 x/2A/2/;; /1 x/1 /+ A/2/;; /2 x/2 /#/:Example /2/./2/./4/. It holds/2/6/6 /6/6/6 /6/4 a a a ba a a ba a a bc c c dc c c d /3/7/7 /7/7/7 /7/5 /2/6/6/6/4 e f fe f fe f fg h h /3/7/7/7/5 /= /"A BC D /#/"E FG H /#/= /"AE /+ BG AF /+ BHCE /+ DG CF /+ DH /#/;;where A /= /( a /) is a /3 / /3 /; matrix/, B /= /( b /) is a /3 / /1 /; matrix/, C /= /( c /) is a/2 / /3 /; matrix/, D /=/( d /)i sa/2 / /1 /; matrix/, E /=/( e /) i sa/3 / /1 /; matrix/, F /=/( f /)is a /3 / /2 /; matrix/, G /=/( g /) i sa/1 / /1 /; matrix and H /=/( h /)i sa/1 / /2 /; matrix/./3/2 Example /2/./2/./5/. /Let us /nd the pro duct AB of blo c k matrices A andB /, when A and B are /3 / /3 /; matricesA /= /2/6/6 /6 /6 /6/6/6/4 /1 /2 /././. /2/3 /4 /././. /0/
/
/
/
/
/
/././. /
/
/
/0 /0 /././. /; /1 /3/7/7 /7 /7 /7/7/7/5 /;; B /= /2/6/6 /6 /6 /6/6/6/4 /; /3 /1 /0 /././. /1/2 /3 /; /1 /././. /1/
/
/
/
/
/
/
/
/
/././. /
/
/
/0 /0 /0 /././. /1 /3/7/7 /7 /7 /7/7/7/5 /:W e denoteA /= /"C DE F /#/;; B /= /"G HK L /#/;;whereC /= /"/1 /2/3 /4 /#/;; D /= /"/2/0 /#/;; E /= h/0 /0 i/;; F /= h/; /1 iandG /= /"/; /3 /1 /0/2 /3 /; /1 /#/;; H /= /"/1/1 /#/;; K /= h/0 /0 /0 i/;; L /= h/1 i/:W e note that the dimensions of the matrices are in accordance with theconditions of m ultiplication of blo c k matrices/. If w e denoteAB /= /"R ST U /#/;;thenR /= CG /+ DK /= /"/1 /2/3 /4 /#/"/; /3 /1 /0/2 /3 /; /1 /#/+ /"/2/0 /#h/0 /0 /0 i/= /"/1 /7 /; /2/; /1 /1/5 /; /4 /#/;;S /= CH /+ DL /= /"/1 /2/3 /4 /#/"/1/1 /#/+ /"/2/0 /#h/1 i/= /"/5/7 /#/;;T /= EG /+ FK /= h/0 /0 i /"/; /3 /1 /0/2 /3 /; /1 /#/+ h/; /1 ih/0 /0 /0 i/= h/0 /0 /0 iandU /= EH /+ FL /= h/0 /0 i /"/1/1 /#/+ h/; /1 ih/1 i/= h/; /1 i/:/3/3 Th usAB /= /2/6/4 /1 /7 /; /2/; /1 /1/5 /; /4 /5/7/0 /0 /0 /; /1 /3/7/5 /= /2/6/4 /1 /7 /; /2 /5/; /1 /1/5 /; /4 /7/0 /0 /0 /; /1 /3/7/5 /:Problem /2/./2/./2/. /Find the pro duct AB of /4 / /5/-matrix A and /5 //4 /; matrix B in blo c k form/, whenA /= /2/6/6/6 /6 /6 /6/6/6 /6/4 /1 /2 /3 /./. /. /0 /0/0 /; /1 /4 /././. /0 /0/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/0 /0 /0 /././. /4 /1/0 /0 /0 /././. /7 /5 /3/7/7/7 /7 /7 /7/7/7 /7/5 /;; B /= /2/6/6/6/6 /6 /6 /6 /6/6/6 /6 /6/4 /1 /; /4 /././. /0 /0/2 /3 /././. /0 /0/5 /; /1 /./. /. /0 /0/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/0 /0 /././. /1 /; /1/0 /0 /././. /4 /; /3 /3/7/7/7/7 /7 /7 /7 /7/7/7 /7 /7/5 /:/1/./2/./3 Determinan tsLet us consider an n / n /; matrix/, the so/-called matrix of or der nA /= /2/6/6 /6/4 a/1/1 a/1/2 /:/:/: a/1 na/2/1 a/2/2 /:/:/: a/2 n/:/:/: /:/:/: /:/:/: /:/:/:an /1 an /2 /:/:/: ann /3/7/7 /7/5 /:De/nition /2/./3/./1/. Arbitrary ordering /1 /;; /2 /;;/:/:/: /;;n of indices i/1 /;;i/2 /;;/:/:/: /;;in iscalled a p ermutation /.De/nition /2/./3/./2/. The ordering of t w o indices in the p erm utation i/1 i/2 /:/:/: inis called natur al if the smaller index stands b efore the greater one/;; in the op/-p osite case/, the greater index standing b efore the smaller one/, it is said thatthe t w o indices form an inversion/.De/nition /2/./3/./3/. A determinant is a la w /(mapping/, function/) thatasso ciates with eac h square matrix A an um b er/, so/-called determinan t of the/3/4 matrixdet /( A /) / /// / / /// / a/1/1 a/1/2 /:/:/: a/1 na/2/1 a/2/2 /:/:/: a/2 n/:/:/: /:/:/: /:/:/: /:/:/:an /1 an /2 /:/:/: ann /// / / /// / /= X/( /; /1/) /a/1 /;;i/1 a/2 /;;i/2 a/3 /;;i/3 /
/
/
an/;;in /;;where the summation go es o v er all the pe r m utations i/1 i/2 i/3 /:/:/: in of indices/1 /;; /2 /;; /3 /;;/:/:/: /;;n and / is the n um b e ro fi n v ersions in the p erm utation i/1 i/2 i/3 /:/:/: inof the ro w indices/. W e will use expressions/: determinan t of order n and itsro ws and columns/.Example /2/./3/./1/. Let us consider the third order determinan tdet/( A /)/= //// / / / a/1/1 a/1/2 a/1/3a/2/1 a/2/2 a/2/3a/3/1 a/3/2 a/3/3 //// / / / /=/=/( /; /1/) /0a/1/1 a/2/2 a/3/3 /+/( /; /1/) /1a/1/1 a/2/3 a/3/2 /+/( /; /1/) /1a/1/2 a/2/1 a/3/3 /+/+/( /; /1/) /2a/1/2 a/2/3 a/3/1 /+/( /; /1/) /2a/1/3 a/2/1 a/3/2 /+/( /; /1/) /3a/1/3 a/2/2 a/3/1 /:Let us examine the last summand /( /; /1/) /3a/1/3 a/2/2 a/3/1 /: In the pe r m utation /3 /2 /1of the column indices the index /3 forms with the index /2 and the index /1 anin v ersion/. The index /2 do es the same with the index /1 /: So the n um be r ofin v ersions / in the p erm utation of the column indces is equal to /3/.Problem /2/./3/./1/. /Whic h sign has the pro ducta/1 /;;n a/2 /;;n /; /1 a/3 /;;n /; /2 /
/
/
an /; /1 /;; /2 an/;; /1of elemen ts of a determinan t expression/.Prop erties of determinan t/ The determinan ts of a matrix and its transp ose are equal/, det/( A T/) /=det/( A /) /:/ Multiplying all the elemen ts of the ro w /(column/) of the determinan tb y the same n um be r the determinan t will be m ultiplied b y the samen um b er/./ In terc hanging t w o ro ws /(columns/) of the determinan tt h e determinan twill c hange its sign/./3/5 / If t w or o ws /(columns/) of the determinan t are iden tical/, then the deter/-minan t is equal to /0/./ If eac h elemen t in the ro w /(column/) of the determinan t is a sum oft w o summands/, then the determinan t expands in to the sum of t w odeterminan ts/, where in the considered ro w /(column/) in the /rst of themthere will b e the /rst summands and in the second of them there willb e the second summands/, and all the remaining ro ws /(columns/) will b eiden tical to those of the giv en matrix/:// /// / / /// / / a/1/1 /:/:/: a/1 n/:/:/: /:/:/: /:/:/:ak /1 /+ bk /1 /:/:/: akn /+ bkn/:/:/: /:/:/: /:/:/:an /1 /:/:/: ann // /// / / /// / / /= // /// / / /// / / a/1/1 /:/:/: a/1 n/:/:/: /:/:/: /:/:/:ak /1 /:/:/: akn/:/:/: /:/:/: /:/:/:an /1 /:/:/: ann // /// / / /// / / /+ // /// / / /// / / a/1/1 /:/:/: a/1 n/:/:/: /:/:/: /:/:/:bk /1 /:/:/: bkn/:/:/: /:/:/: /:/:/:an /1 /:/:/: ann // /// / / /// / / /:/ The determinan t will not c hange if an arbitrary ro w /(column/) m ulti/-plied b y an arbitrary n um be r isadded to ar o w /(column/)/./ The fundamen tal form ulas of the determinan t theory /(or the or em ofexp ansion by c ofactors /) are v alid/:ai /1 Ak /1 /+ ai /2 Ak /2 /+ /:/:/: /+ ain Akn /=d e t /( A /) /
/ik /;;a/1 i A/1 k /+ a/2 i A/2 k /+ /:/:/: /+ ani Ank /=d e t /( A /) /
/ik /;;where/ik /= /(/1 /;; as i /= k/0 /;; as i /6/= kis the Kronec k er sym b ol and Aik is the pro duct of the n um be r /( /; /1/) i /+ kand the determinan t of the /( n /; /1/) / /( n /; /1/) /; matrix obtained from thegiv en matrix b y deleting the i /-th ro w and k /-th column/.Example /2/./3/./2/. Let us ev aluate the determinan t of order n /, using theexpansion b y cofactors b y the /rst column and then b y the /rst ro w/.Dn /= // /// / ///// / / / /; /2 /1 /0 /:/:/: /0 /0/1 /; /2 /1 /:/:/: /0 /0/0 /1 /; /2 /:/:/: /0 /0/:/:/: /:/:/: /:/:/: /:/:/: /:/:/: /:/:/:/0 /0 /0 /:/:/: /; /2 /1/0 /0 /0 /:/:/: /1 /; /2 // /// / ///// / / / /=/3/6 /=/( /; /2/)/( /; /1/) /1/+/1Dn /; /1 /+/1 /
/( /; /1/) /2/+/1 // / / /// / / / // /1 /0 /:/:/: /0 /0/1 /; /2 /:/:/: /0 /0/:/:/: /:/:/: /:/:/: /:/:/: /:/:/:/0 /0 /:/:/: /; /2 /1/0 /0 /:/:/: /1 /; /2 // / / /// / / / // /=/= /; /2 Dn /; /1 /; Dn /; /2orDn /+/2 Dn /; /1 /+ Dn /; /2 /=/0 /: /(/1/)Equation /(/1/) is a linear homogeneous di/erence equation with constan t co/-e/cien ts whic h has the solution of t yp e / n/: Let us try to /nd them/:/ n/+/2 / n /; /1/+ / n /; /2/=/0 /, / n /; /2/( / /2/+/2 / /+/1 /) /= /0 /:W e are in terested in a non/-trivial solution/. So w e ha v e get a quadraticequation/ /2/+/2 / /+/1 /= /0 /;;to /nd the solution of the di/erence equation /(/1/)/. It has the solutions//1 /;; /2 /= /; /1/, and so one of the solutions of equation /(/1/) is Dn /= /( /; /1/) n/: Asthe n um be r /; /1 is a double solution of the quadratic equation/, Dn /=/( /; /1/) nnwill be a solution of the equation /(/1/)/, to o/. Th us/, w e ha v e got t w o linearlyindep enden t particular solutions of the linear homogeneous di/erence equa/-tion with constan t co e/cien ts/. The general solution of the equation can beexpressed in formDn /= C/1 /( /; /1/) n/+ C/2 /( /; /1/) nn/:F rom the conditions D/1 /= /; /2 and D/2 /= /3 w e can /nd the co e/cien ts C/1and C/2 /:/(C/1 /( /; /1/) /1/+ C/2 /( /; /1/) /1/
/1/= /; /2C/1 /( /; /1/) /2/+ C/2 /( /; /1/) /2/
/2/= /3 /) /(C/1 /=/1C/2 /=/1So the giv en problem has the solutionDn /=/( /; /1/) n/( n /+/1 /) /:/3/7 Problem /2/./3/./2/. /Compute the determinan to fo r d e r n// / / /// / / / /// / / /// /7 /5 /0 /
/
/
/0 /0/2 /7 /5 /./././0 /0/0 /2 /7 /././. /./././0/././. /././. /././. /././. /././. /./././0 /0 /././. /./././7 /5/0 /0 /0 /./././2 /7 // / / /// / / / /// / / /// /:Example /2/./3/./3/. Ev aluate the V andermonde determinan tVn /( x/1 /;;x/2 /;;/:/:/: /;;xn /)/= //// / / /// / / /// /1 /1 /1 /:/:/: /1x/1 x/2 x/3 /:/:/: xnx /2/1 x /2/2 x /2/3 /:/:/: x /2n/:/:/: /:/:/: /:/:/: /:/:/: /:/:/:x n /; /2/1 x n /; /2/2 x n /; /2/3 /:/:/: x n /; /2nx n /; /1/1 x n /; /1/2 x n /; /1/3 /:/:/: x n /; /1n //// / / /// / / /// /:W e substract x/1 times the p en ultimate ro w from the last ro w/, then x/1 timesthe /( n /; /2/) /; th ro w from the p en ultimate ro w/, then x/1 times /( n /; /3/) /; th ro wfrom the /( n /; /2/) /; th ro w etc/./, in the end x/1 times the second ro w from the/rst one/. As a result/, w e get/= // /// / / /// / / / / /1 /1 /1 /:/:/: /1/0 x/2 /; x/1 x/3 /; x/1 /:/:/: xn /; x/1/0 x /2/2 /; x/1 x/2 x /2/3 /; x/1 x/3 /:/:/: x /2n /; x/1 xn/:/:/: /:/:/: /:/:/: /:/:/: /:/:/:/0 x n /; /2/2 /; x/1 x n /; /3/2 x n /; /2/3 /; x/1 x n /; /3/3 /:/:/: x n /; /2n /; x/1 x n /; /3n/0 x n /; /1/2 /; x/1 x n /; /2/2 x n /; /1/3 /; x/1 x n /; /2/3 /:/:/: x n /; /1n /; x/1 x n /; /2n // /// / / /// / / / / /:Using the expression b y the /rst column and factoring out the common fac/-tors in the elemen ts/, w e get/= // /// / / / /// / x/2 /; x/1 x/3 /; x/1 /:/:/: xn /; x/1x/2 /( x/2 /; x/1 /) x/3 /( x/3 /; x/1 /) /:/:/: xn /( xn /; x/1 /)/:/:/: /:/:/: /:/:/: /:/:/:x n /; /3/2 /( x/2 /; x/1 /) x n /; /3/3 /( x/3 /; x/1 /) /:/:/: x n /; /3n /( xn /; x/1 /)x n /; /2/2 /( x/2 /; x/1 /) x n /; /2/3 /( x/3 /; x/1 /) /:/:/: x n /; /2n /( xn /; x/1 /) // /// / / / /// / /:/3/8 F actoring out from the /rst columns the common factor x/2 /; x/1 /, from thesecond column x/3 /; x/1 /;; /:/:/: /, from the /( n /; /1/) /; th column xn /; x/1 /;; w e get/=/( x/2 /; x/1 /)/( x/3 /; x/1 /) /
/
/
/( xn /; x/1 /) // / ///// / /// /1 /1 /:/:/: /1x/2 x/3 /:/:/: xnx /2/2 x /2/3 /:/:/: x /2n/:/:/: /:/:/: /:/:/: /:/:/:x n /; /2/2 x n /; /2/3 /:/:/: x n /; /2n // / ///// / /// /:Using the same op erations cycle/, results inVn /( x/1 /;;x/2 /;;/:/:/: /;;xn /)/= Yn / k />i / /1 /( xk /; xi /) /:Prop osition /2/./3/./1 /( The Laplace expansion theorem/) /. The so/-calledLaplace form uladet/( A /)/= XMk An /; kholds/, where the summation on the righ tg o e s o v er all determinan ts /(minors/)Mk of order k that can be formed of ro ws i/1 /, i/2 /, /:/:/: /, ik and columns j/1 /;; j/2 /;;/:/:/: /, jk /, and An /; k is the pro duct of the n um be r /( /; /1/) i/1 /+ i/2 /+ /:/:/: /+ ik /+ j/1 /+ j/2 /+ /:/:/: /+ jkand the determinan t of the matrix remaining from the matrix A b y deletingthe ro ws i/1 /, i/2 /, /:/:/: /, ik and the columns j/1 /;; j/2 /;; /:/:/: /, jk used in forming theminor Mk /.Pr o of/. See Kangro /(/1/9/6/2/, pp/. /3/7/-/3/9/)/. /2Example /2/./3/./4/. Using the Laplace expansion b y the /rst t w o ro ws/,transform the determinan t// / /// / / / a b c /0d e f /0/0 a b c/0 d e f // / /// / / / /:As only three minors are not equal to zero /, w e get the expansion// / /// / // a b c /0d e f /0/0 a b c/0 d e f // / /// / // /=/( /; /1/) /1/+/2/+/1/+/2 //// / a bd e //// / /
//// / b ce f //// / /+/3/9 /+/( /; /1/) /1/+/2/+/1/+/3 // / / / a cd f // / / / /
// / / / a cd f // / / / /+/( /; /1/) /1/+/2/+/2/+/3 // / / / b ce f // / / / /
// / / / /0 c/0 f // / / / /:Problem /2/./3/./3/. /Compute b y the use of the Laplace form ula the deter/-minan t/// / / /// / / // /0 /0 /0 /2 /; /1/0 /0 /1 /5 /3/0 /0 /0 /2 /3/; /1 /1 /3 /1 /2/2 /2 /0 /0 /3 /// / / /// / / // /:By the Laplace expansion theorem/, it holds for eac h matrix C /= /2/6/4 c/1/1 /:/:/: c/1 n/:/:/: /:/:/: /:/:/:cn /1 /:/:/: cnn /3/7/5the equalit y// /// / / /// / / // a/1/1 /:/:/: a/1 n /0 /:/:/: /0/:/:/: /:/:/: /:/:/: /:/:/: /:/:/: /:/:/:an /1 /:/:/: ann /0 /:/:/: /0c/1/1 /:/:/: c/1 n b/1/1 /:/:/: b/1 n/:/:/: /:/:/: /:/:/: /:/:/: /:/:/: /:/:/:cn /1 /:/:/: cnn bn /1 /:/:/: bnn // /// / / /// / / // /= // / / /// a/1/1 /:/:/: a/1 n/:/:/: /:/:/: /:/:/:an /1 /:/:/: ann // / / /// /
// / / /// b/1/1 /:/:/: b/1 n/:/:/: /:/:/: /:/:/:bn /1 /:/:/: bnn // / / /// /(/2/)Cho osing C /= /2/6/4 /; /1 /:/:/: /0/:/:/: /:/:/: /:/:/:/0 /:/:/: /; /1 /3/7/5 /;; w e transform the determinan t// / /// / / / /// / / a/1/1 /:/:/: a/1 n /0 /:/:/: /0/:/:/: /:/:/: /:/:/: /:/:/: /:/:/: /:/:/:an /1 /:/:/: ann /0 /:/:/: /0/; /1 /:/:/: /0 b/1/1 /:/:/: b/1 n/:/:/: /:/:/: /:/:/: /:/:/: /:/:/: /:/:/:/0 /:/:/: /; /1 bn /1 /:/:/: bnn // / /// / / / /// / /so that all the elemen ts bij b ecome zeros/. T o mak e b/1/1 /;;b/2/1 /;;/:/:/: /;;bn /1 in to zerosw e ha v e to add to the /( n /+ /1/)/-th column b/1/1 times the elemen ts of the /rstcolumn/, b/2/1 times the elemen ts of the second column etc/, and/, in the end/, bn /1times the elemen ts of the n /-th column/. Next w e mak ei n to zeros the elemen tsb/1/2 /;;b/2/2 /;;/:/:/: /;;bn /2 /: F or this w e add to the /( n /+/2 /) /; th column b/1/2 times the /rstcolumn/, b/2/2 times the second column etc/, and/, in the end/, bn /2 times the n /-th/4/0 column etc/. The last step will n ullify the elemen ts b/1 n /;;b/2 n /;;/:/:/: /;;bnn /: F or thisw e add to the /2 n /; th column b/1 n times the /rst column/, b/2 n times the secondcolumn etc/, and/, in the end/, bnn times the n /-th column/. The result will b e// /// / / /// / / / / a/1/1 /:/:/: a/1 n /0 /:/:/: /0/:/:/: /:/:/: /:/:/: /:/:/: /:/:/: /:/:/:an /1 /:/:/: ann /0 /:/:/: /0/; /1 /:/:/: /0 b/1/1 /:/:/: b/1 n/:/:/: /:/:/: /:/:/: /:/:/: /:/:/: /:/:/:/0 /:/:/: /; /1 bn /1 /:/:/: bnn // /// / / /// / / / / /= // /// / / /// / / / / a/1/1 /:/:/: a/1 n d/1/1 /:/:/: d/1 n/:/:/: /:/:/: /:/:/: /:/:/: /:/:/: /:/:/:an /1 /:/:/: ann dn /1 /:/:/: dnn/; /1 /:/:/: /0 /0 /:/:/: /0/:/:/: /:/:/: /:/:/: /:/:/: /:/:/: /:/:/:/0 /:/:/: /; /1 /0 /:/:/: /0 // /// / / /// / / / / /=/=/( /; /1/) n /+/1/+ n /+/2/+ /:/:/: /2 n /+/1/+/2/+ /:/:/: n // / / /// /; /1 /:/:/: /0/:/:/: /:/:/: /:/:/:/0 /:/:/: /; /1 // / / /// /
// / / /// d/1/1 /:/:/: d/1 n/:/:/: /:/:/: /:/:/:dn /1 /:/:/: dnn // / / /// /=/=/( /; /1/) /(/1/+/2 n /)/2 n /2 /+ n // / / /// d/1/1 /:/:/: d/1 n/:/:/: /:/:/: /:/:/:dn /1 /:/:/: dnn // / / /// /= // / / /// d/1/1 /:/:/: d/1 n/:/:/: /:/:/: /:/:/:dn /1 /:/:/: dnn // / / /// /;;wheredij /= nXk /=/1 aik bkj /: /(/3/)T aking in to accoun t /(/2/) and the fact that/, b y /(/3/)/, D /= A /
B /, w e reac h theassertion/.Prop osition /2/./3/./1 /(the theorem ab out the determinan t of the pro ductof matrices/)/. F or arbitrary matrices A and B of order n it holdsdet/( AB /) /= /(det A /)/(det B /) /:/1/./2/./4 F our Subspaces of a MatrixLet us consider an m / n /; matrixA /= /2/6/6 /6/4 a/1/1 a/1/2 /:/:/: a/1 na/2/1 a/2/2 /:/:/: a/2 n/:/:/: /:/:/: /:/:/: /:/:/:am /1 am /2 /:/:/: amn /3/7/7 /7/5/4/1 with real elemen ts/. The matrix A can be expressed bo t h b y the column/-v ectors ck /( k /= /1 /: n/) of A or b y the ro w/-v ectors r Ti /( i /=/1 /: m/) b y thetransp ose of AA /= hc/1 /
/
/
cn i/= hc/1 /;; /
/
/
/;; cn i/= /2/6/6/4 r T/1/././.r Tm /3/7/7/5 /;;where ri /2 R nand ck /2 R m/;; and ri /= /2/6/6/4 ai /1/./. /.ain /3/7/7/5 /;; ck /= /2/6/6/4 a/1 k/./. /.amk /3/7/7/5 /:De/nition /2/./4/./1/. The subspace span f c/1 /;;/:/:/: /;; cn g of the set f c/1 /;;/:/:/: /;; cn gof column/-v ectors of the matrix A is called the subsp ac eo fc olumn/-ve ctors ofthe matrix A /, and denoted b y R /( A /) or ra n /( A /) /:De/nition /2/./4/./2/. The subspace span f r/1 /;;/:/:/: /;; rm g of the set f r/1 /;;/:/:/: /;; rm gof the ro w/-v ectors of the matrix A is called the subsp ac e of r ow/-ve ctors ofthe matrix A /, and denoted b y R /( A T/)o r ra n /( A T/) /:De/nition /2/./4/./3/. The r ank of the matrix A is the greatest naturaln um be r k/;; for whic h there exist a minor of order k di/eren t from zero/. W edenote the rank of A b y ra n k /( A /) /:Let r ank /( A /) /= r /: Due to the theorem ab out the rank of the matrix/, w egetProp osition /2/./4/./1 /: The rank of the matrix is equal to the dimension ofthe subspace of its ro w/-v ectors or column/-v ectors/, i/.e/./,ra n k /( A /) /= dim R /( A T/) /= dim R /( A /)/= r /:De/nition /2/./4/./4/. The /(right/) nul l sp ac e of the matrix A is the set of allsolutionsx /= /2/6/4 //1/:/:/:/n /3/7/5 /= h//1 /:/:/: /n iTof the system of equationsA x /=/0 /: /(/4/)It is a subspace/, denoted N /( A /)o r nul l /( A /) /:/4/2 Prop osition /2/./4/./2/. F or ev ery matrix A /2 R m / nwith the rank r /;;dim N /( A /)/= n /; r /^ N /( A /) /?R /( A T/) /^ N /( A /) /R /( A T/)/= R n/:Pr o of/. The matrix of the system has the rank r /, and the n um be r ofv ariables in /(/4/) equals n/: Therefore/, the n um b er of degrees of freedom of thesystem is n /; r /: The n um b er of degrees of freedom giv es the dimension of then ull space/. Th us/, dim N /( A /)/= n /; r /: W e can rewrite the system /(/4/) in form/2/6/4 r T/1 x/:/:/:r Tm x /3/7/5 /= /2/6/4 /0/:/:/:/0 /3/7/5 /:Therefore/, r Tk x /= /0 /, rk /? x /( k /= /1 /: m/) /;; i/.e/./, the ro w/-v ectors of A areorthogonal to an y v ector of the n ull space N /( A /) of the matrix A /. HenceN /( A /) /? R /( A T/) /: As/, in addition/, dim N /( A /) /= n /; r and dim R /( A T/) /= r /;;dim N /( A /)/+d i m R /( A T/)/= n and the space R ncan b e expressed b y the directsumR n/= N /( A /) /R /( A T/) /: /2De/nition /2/./4/./5/. The /(left/) nul l sp ac e of the matrix A is the set of allsolutionsy /= /2/6/4 //1/:/:/:/m /3/7/5 /= h//1 /:/:/: /m iTof the system of equationsA Ty /=/0 /(/5/)This subspace is denoted b y N /( A T/)o r nul l /( A T/) /:Prop osition /2/./4/./3/. F or ev ery matrix A /2 R m / nwith the rank r /;;dim N /( A T/)/= m /; r /^ N /( A T/) /?R /( A /) /^ N /( A T/) /R /( A /)/= R m/:Pr o of/. The matrix of the system A Thas the rank r /;; and/, the n um be r ofv ariables in /(/5/) equals m/: Therefore/, the n um b er of degrees of freedom of thesystem is m /; r anddim N /( A T/)/= m /; r /:/4/3 The system /(/5/) can be expressed in form/2/6/4 c T/1 y/:/:/:c Tm y /3/7/5 /= /2/6/4 /0/:/:/:/0 /3/7/5 /:So c Tk y /= /0 /, ck /? y /( k /= /1 /: m/) and N /( A T/) /? R /( A /) /: As dim N /( A T/) /=m /; r and dim R /( A /) /= r /;; dim N /( A T/) /+ dim R /( A /) /= m/;; and the space R mcan b e expressed b y the direct sumR m/= N /( A T/) /R /( A /) /: /2Example /2/./4/./1/. Let us /nd the dimensions and bases of the subspacesR /( A /) /;; N /( A /) /;; R /( A T/) and N /( A T/) for the matrixA /= /2/6/4 /1 /2 /0 /1 /1/0 /1 /1 /0 /1/1 /2 /0 /1 /1 /]W e will illustrate the assertion of prop ositions /2/./4/./2 and /2/./4/./3 in case of thisexample/.W e start with the examination of the space R /( A /)/. Substituting from thesecond column of A t w o times the /rst column/, w e get/2/6/4 /1 /2 /0 /1 /1/0 /1 /1 /0 /1/1 /2 /0 /1 /1 /3/7/5 / /2/6/4 /1 /0 /0 /1 /1/0 /1 /1 /0 /1/1 /0 /0 /1 /1 /3/7/5 / /;;then substracting from the third column the new second one/, from the fourthcolumn the /rst one and from the /fth column the /rst one and the newsecond one/, w e get/ /2/6/4 /1 /0 /0 /0 /0/0 /1 /0 /0 /0/1 /0 /0 /0 /0 /3/7/5 /:The sym bo l /" / /" be t w een the matrices marks that R /( A /) is not c hanged/.The last matrix has only t w o columns di/eren t from the n ull v ector /)dim R /( A /)/=/2 /: The basis in the space R /( A /) will b eSR /( A /) /= f /2/6/4 /1/0/1 /3/7/5 /;; /2/6/4 /0/1/0 /3/7/5 g /:/4/4 T o describ e the space N /( A T/)/, w e solv e system /(/5/)/:/2/6/6 /6/6/6 /6 /6 /6/6/6/4 /1 /0 /1 /././. /0/2 /1 /2 /././. /0/0 /1 /0 /././. /0/1 /0 /1 /./. /. /0/1 /1 /1 /./. /. /0 /3/7/7 /7/7/7 /7 /7 /7/7/7/5 / /2/6/6 /6/6/6 /6 /6 /6/6/6/4 /1 /0 /1 /././. /0/0 /1 /0 /././. /0/0 /0 /0 /././. /0/0 /0 /0 /./. /. /0/0 /0 /0 /./. /. /0 /3/7/7 /7/7/7 /7 /7 /7/7/7/5 /;;i/.e/./,/(//1 /+/0 //2 /+ //3 /= /0//2 /= /0 /) //2 /=/0 /^ //3 /= p /^ //1 /= /; p /)y /= /2/6/4 /; p/0p /3/7/5 /= p /2/6/4 /; /1/0/1 /3/7/5 /) dim N /( A T/)/= /1 /^ SN /( A T/) /= f /2/6/4 /; /1/0/1 /3/7/5 g /:Let us c hec k b y scalar pro duct that SR /( A /) /? SN /( A T/) /:h/1 /0 /1 i/
/2/6/4 /; /1/0/1 /3/7/5 /=/1 /
/( /; /1/) /+ /0 /
/0/+/1 /
/1/= /0 /;;h/0 /1 /0 i/
/2/6/4 /; /1/0/1 /3/7/5 /=/0 /:The union SR /( A /) /[ SN /( A T/) con tains three linearly indep enden tv ectors of R /3/.These v ectors form a basis in R /3/. Th us/, R /3/= N /( A T/) /R /( A /) /: T o describ ethe space R /( A T/)l e t us /nd its dimension and basis/:/2/6/4 /1 /2 /0 /1 /1/0 /1 /1 /0 /1/1 /2 /0 /1 /1 /3/7/5 / /2/6/4 /1 /2 /0 /1 /1/0 /1 /1 /0 /1/0 /0 /0 /0 /0 /3/7/5 /)dim R /( A T/)/= /2 /^ SR /( A T/) /= f /2/6/6 /6 /6 /6 /6/4 /1/2/0 /1/1 /3/7/7 /7 /7 /7 /7/5 /;; /2/6/6 /6 /6 /6 /6/4 /0/1/1 /0/1 /3/7/7 /7 /7 /7 /7/5 g /:/4/5 T o describ e the space N /( A /)/, w e solv e system /(/4/)/:/2/6/6 /6/4 /1 /2 /0 /1 /1 /././. /0/0 /1 /1 /0 /1 /./. /. /0/1 /2 /0 /1 /1 /././. /0 /3/7/7 /7/5 / /2/6/6 /6/4 /1 /2 /0 /1 /1 /././. /0/0 /1 /1 /0 /1 /./. /. /0/0 /0 /0 /0 /0 /././. /0 /3/7/7 /7/5 /)/(//1 /+/2 //2 /+/0 //3 /+ //4 /+ //5 /=/0//2 /+ //3 /+/0 //4 /+ //5 /=/0 /) //3 /= p/;; //4 /= q/;; //5 /= t//2 /= /; p /; t/;; //1 /=/2 p /; q /+ t /)x /= /2/6/6/6 /6 /6 /6/4 /2 p /; q /+ t/; p /; tpqt /3/7/7/7 /7 /7 /7/5 /= p /2/6/6/6 /6 /6 /6/4 /2/; /1/1/0/0 /3/7/7/7 /7 /7 /7/5 /+ q /2/6/6/6 /6 /6 /6/4 /; /1/0/0/1/0 /3/7/7/7 /7 /7 /7/5 /+ q /2/6/6/6 /6 /6 /6/4 /1/; /1/0/0/1 /3/7/7/7 /7 /7 /7/5 /)SN /( A /) /= f /2/6/6 /6 /6/6/6/4 /2/; /1/1/0 /0 /3/7/7 /7 /7/7/7/5 /;; /2/6/6 /6 /6/6/6/4 /; /1/0/0/1 /0 /3/7/7 /7 /7/7/7/5 /;; /2/6/6 /6 /6/6/6/4 /1/; /1/0/0 /1 /3/7/7 /7 /7/7/7/5 g/) dim SN /( A /) /=/3 /:The v ectors of the basis SN /( A /) are orthogonal to the v ectors of the basisSR /( A T/) /: Th us/, R /( A T/) /?N /( A /) and the union SR /( A T/) /[ SN /( A /) forms a basis inR /5/: ThereforeN /( A T/) /R /( A /)/= R /5/:Problem /2/./4/./1/. Let A /2 R n / n/: Sho w that N /( A TA /)/= N /( A /) /:Problem /2/./4/./2/. Sho w thatN /( AB /) / N /( B /) /^ N /(/( AB /) T/) /N /( A T/) /^R /( AB /) / R /( A /) /^ R /(/( AB /) T/) /R /( B T/) /:Problem /2/./4/./3/. /Find the dimensions and bases of the subspaces R /( A /) /;;N /( A /) /;; R /( A T/) and N /( A T/) of the matrix A /. Demonstrate the assertion ofprop osition /2/./4/./2 and /2/./4/./3 on the matrix A /,w h e r ea /) A /= /2/6/4 /; /2 /; /2 /; /1/0 /1/3 /2 /1/2 /; /1/; /1 /; /1 /; /5 /1 /3/7/5 /;; b /) A /= /2/6/4 /1 /; /1 /; /1 /; /1/0 /1 /3 /5/; /2 /2 /2 /2 /3/7/5 /;;/4/6 c /) A /= /2/6/6 /6/4 /8 /1/6 /2 /6/2 /4 /; /1 /3/9 /1/8 /2 /7/3 /6 /0 /3 /3/7/7 /7/5 /;; d /) A /= /2/6/6 /6/4 /1 /2 /; /1 /2 /; /2/; /1 /; /2 /2 /; /3 /3/; /1 /; /2 /0 /; /1 /1/; /2 /; /4 /0 /; /2 /2 /3/7/7 /7/5 /:Problem /2/./4/./4/. /Find the dimensions and bases of the subspaces R /( AB /) /;;N /( AB /) /;; R /(/( AB /) T/) and N /(/( AB /) T/) of the pro duct AB /, whereA /= /2/6/4 /1 /; /1 /; /1 /; /1/0 /1 /3 /5/; /2 /2 /2 /2 /3/7/5 /^ B /= /2/6/6 /6/4 /8 /1/6 /2 /6/2 /4 /; /1 /3/9 /1/8 /2 /7/3 /6 /0 /3 /3/7/7 /7/5 /:Compare the results obtained with the results of Problem /2 /: /4 /: /3 /in case b /)and c /)/./1/./2/./5 Eigen v alues and Eigen v ectors of a MatrixDe/nition /2/./5/./1/. IfA x /= / x /;; /(/6/)where A /2 C n / n/, x /2 C nand / is a n um b er/, then the n um be r / is calledan eigenvalue of the matrix A and the v ector x a /(right/) eigenve ctor of thematrix A corresp onding to the eigen v alue / /.De/nition /2/./5/./2/. The v ector x is called a /(left/) eigenve ctor of the matrixA if x HA /= / x H/;; where x His the transp osed sk ew/-matrix/.Prop osition /2/./5/./1/. If x is a left eigen v ector of the matrix A corresp ond/-ing to the eigen v alue / /, then this x is a righ t eigen v ector corresp onding tothe eigen v alue / /.Pr o of/. W e get ac hain of assertions/:x HA /= / x H/, /( x HA /) H/=/( / x H/) H/, A Hx /= / x /: /2It is ob vious that if x is a eigen v ector corresp onding to the eigen v alue / /,then c x /;; c /2 C is an eigen v ector/, to o/. The equation /(/6/) can be expressed inform/( A /; /I /) x /=/0 /;; /(/7/)/4/7 where I is the iden tit y matrix of order n /. As the n ull v ector is an eigen v ectorfor ev ery square matrix A in eigen v alues problem /(/6/)/, in follo wing w e willcon/ne ourselv es to the non/-trivial eigen v ectors/. The equation /(/7/) presen ts asystem of homogeous linear algebraic equations that has a non/-trivial solution i/ the matrix A /; /I of the system is singular/, i/.e/./,det /( A /; /I /)/= /0 /: /(/8/)The equation /(/8/) is called the char acteristic e quation of the matrix A /, andthe p olynomialp /( / /)/= det /( A /; /I /)is called the char acteristic p olynomial of the matrix A /. The equation /(/8/) isan algebraic equation of order n with resp ect to / /, and it can be writtendo wn in form/:// /// / / // a/1/1 /; / a/1/2 /
/
/
a/1 na/2/1 a/2/2 /; / /
/
/
a/2 n/
/
/
/
/
/
/
/
/
/
/
/
an /1 an /2 /
/
/
ann /; / // /// / / // /= /0 /: /(/9/)According to the fundamen tal theorem of algebra/, the matrix A /2 C n / nhasexactly n eigen v alues/, taking in to accoun t their m ultiplicit y /.De/nition /2/./5/./3/. The set of all eigen v alues f //1 /;;/:/:/: /;;/n g of the matrixA /2 C n / nis called the sp e ctrum of the matrix A and denoted b y / /( A /) /:Example /2/./5/./1 /. Find the eigen v alues an d eigen v ectors of the matrixA /= /2/6/4 /1 /1 /1/1 /1 /1/1 /1 /1 /3/7/5W e comp ose the c haracteristic equation /(/9/) corresp onding to the giv en ma/-trix/:// / /// / /1 /; / /1 /1/1 /1 /; / /1/1 /1 /1 /; / // / /// / /= /0 /:Calculating the determinan t/, w e get the cubic equation/(/1 /; / /) /3/; /3/(/1 /; / /) /+ /2 /= /0 /;;/4/8 with the solutions //1 /= //2 /= /0 and //3 /= /3 /: Let us /nd the eigen v ectorscorresp onding to the eigen v alues //1 /= //2 /=/0 /. W e replace in system /(/7/) thev ariable / b y /0 and solv e the equation/:/2/6/6 /6/4 /1 /; /0 /1 /1 /././. /0/1 /1 /; /0 /1 /././. /0/1 /1 /1 /; /0 /././. /0 /3/7/7 /7/5 / /2/6/6 /6/4 /1 /1 /1 /././. /0/0 /0 /0 /././. /0/0 /0 /0 /././. /0 /3/7/7 /7/5 /:There is only one indep enden t equation remained/://1 /+ //2 /+ //3 /=/0 /:The n um b er of degrees of freedom of the system is /2/, and the general solutionof the system isx /= /2/6/4 //1//2//3 /3/7/5 /= /2/6/4 /; q /; pqp /3/7/5 /= p /2/6/4 /; /1/0/1 /3/7/5 /+ q /2/6/4 /; /1/1/0 /3/7/5 /;;where p and q are arbitrary real n um b ers/. Th us/, the v ectors x that corre/-sp ond to the eigen v alues //1 /= //2 /= /0 form a t w o/-dimensional subspace inthe space R /3/, and v ectors x/1 /= /[ /; /1 /0 /1/] Tand x/2 /= /[ /; /1 /1 /0/] Tcan bec hosen for its basis/. T o /nd the eigen v ector corresp onding to the eigen v alue//3 /=/3 w eh a v e to replace in the system of equations /(/7/) the v ariable / b y/3 /:As a result/, w e get the system of equations/:/2/6/6 /6/4 /1 /; /3 /1 /1 /././. /0/1 /1 /; /3 /1 /././. /0/1 /1 /1 /; /3 /././. /0 /3/7/7 /7/5 / /2/6/6 /6/4 /1 /1 /; /2 /././. /0/1 /; /2 /1 /././. /0/; /2 /1 /1 /././. /0 /3/7/7 /7/5 // /2/6/6 /6/4 /1 /1 /; /2 /././. /0/0 /; /3 /3 /././. /0/0 /3 /; /3 /././. /0 /3/7/7 /7/5 / /2/6/6 /6/4 /1 /1 /; /2 /././. /0/0 /1 /; /1 /././. /0/0 /0 /0 /././. /0 /3/7/7 /7/5 / /2/6/6 /6/4 /1 /0 /; /1 /././. /0/0 /1 /; /1 /././. /0/0 /0 /0 /././. /0 /3/7/7 /7/5 /:The n um be r of degrees of freedom of this system is /1/, and the eigen v ectorsof the matrix A corresp onding to the eigen v alue //3 /=/3 can be expressed informx /= /2/6/4 rrr /3/7/5 /= r /2/6/4 /1/1/1 /3/7/5 /:/4/9 They form a one/-dimensional subspace in R /3with the basis v ector x/3 /=/[/1 /1 /1/] T/:Problem /2/./5/./1 /. /Find the eigen v alues and eigen v ectors of the matrix A /,wherea /) A /= /"/2 /3/; /1 /6 /#/;; b /) A /= /"/5 /; /2/7 /4 /#/;; c /) A /= /"/1 /; /1/2 /4 /#/:Problem /2/./5/./2 /. /Find the eigen v alues and eigen v ectors of the matrix A /,whena /) A /= /2/6/4 /3 /; /2 /2/0 /1 /0/; /1 /1 /0 /3/7/5 /;; b /) A /= /2/6/4 /1 /1 /1/0 /1 /1/0 /0 /0 /3/7/5 /:Prop osition /2/./5/./2/. If //1 /;;//2 /;; /
/
/
/;;/n are the eigen v alues of the matrix A /,thendet /( A /)/= //1 //2 /
/
/
/n /:Pr o of/. The left side of the c haracteristic equation /(/8/) with the zeros//1 /;; /
/
/
/;;/n can b e expressed in formdet /( A /; /I /)/= /( /; /1/) n/( / /; //1 /) /
/
/
/( / /; /n /) /: /(/1/0/)If w e tak e in this equation / /=/0 /;; w e get the assertion of the prop osition/. /2Corollary /2/./5/./1/. Not a single one of the eigen v alues of a regular matrixA is equal to /0 /:Prop osition /2/./5/./3/. If x is an eigen v ector of the regular matrix A corre/-sp onding to the eigen v alue / /, then the same v ector x is as eigen v ector of thein v erse matrix A /; /1corresp onding to the eigen v alue /1 /=/ /.T o pr ove the assertion w e m ultiply the b oth sides of the equalit y /(/6/) onthe left b y the matrix A /; /1/: W e get A /; /1A x /= A /; /1/ x or A /; /1x /=/( /1 /=/ /) x /: /2Prop osition /2/./5/./4/. If x is an eigen v ector of the matrix A corresp ondingto the eigen v alue / /, then the same v ector x is an eigen v ector of the matrixA /2corresp onding to the eigen v alue / /2/.Pr o of/. This assertion follo ws from the c hain/:A /2x /= A /( A x /)/= A /( / x /)/= / /( A x /)/= / /( / x /)/= / /2x /: /2/5/0 Problem /2/./5/./3 /. /Let //1 /;;/:/:/: /;;/n be the eigen v alues of the matrix A /2C n / n/: Pro v e that / k/1 /;;/:/:/: /;;/ kn are the eigen v alues of the matrix A k/( k /2 N /)/.Problem /2/./5/./4 /. /Pro v e that if //1 /;;/:/:/: /;;/n are the eigen v alues of the matrixA /2 C n / n/,t h e n //1 / / /;;/:/:/: /;;/n / / are the eigen v alues of the matrix A / /I /.Prop osition /2/./5/./5/. The trace of the matrix A /, i/.e/./, the sum of theelemen ts on the main diagonal/, is equal to the sum of all eigen v alues of thematrix A /.T o pr ove the assertion w e will use the equalit y /(/1/0/)/. In the expansion ofthe left side b yt h ep o w ers of the v ariable / the co e/cien tb y the p o w er / n /; /1is /( /; /1/) n /; /1/( a/1/1 /+ a/2/2 /+ /
/
/
/+ ann /) and at the righ ts i d e i ti s /( /; /1/) n /+/1/( //1 /+ //2 /+/
/
/
/+ /n /) /: /2Example /2/./5/./2 /. /Supp ose w e kno w three eigen v alues //1 /=/4 /;; //2 /= /1 and//3 /=/6 of the matrixA /= /2/6/6 /6/4 /4 /2 /0 /4/0 /2 /; /1 /0/0 /0 /3 /3/0 /4 /0 /7 /3/7/7 /7/5Let us /nd the forth eigen v alue of the matrix A and its determinan t/. Sincethe trace of the matrix A equals the sum af all eigen v alues/,/4/+/2/+/3/+/7 /= /4/+/1/+/6 /+ //4 /) //4 /=/5 /:Computing the determinan t/, w e getdet/( A /)/= //1 //2 //3 //4 /=/4 /
/1 /
/6 /
/5 /= /1/2/0 /:Problem /2/./5/./5 /. /Supp ose w e kno w three eigen v alues //1 /= /7 /;; //2 /= /; /7and //3 /=/2 /1 o f t h e matrixA /= /2/6/6/6/4 /6/7 /2/6/6 /; /3/0 /6/4/; /2/4 /; /9/1 /1/2 /; /2/0/; /6 /; /4/2 /1/0 /; /1/2/4/2 /1/2/6 /; /2/1 /2/1 /3/7/7/7/5 /:Find the forth eigen v alue of the matrix A and its determinan t/.Prop osition /2/./5/./6/. The eigen v alues of b oth an upp er triangular or alo w er triangular matrix are the elemen ts of the main diagonal/./5/1 Pr o of/. Let us consider the case of an upp er triangular matrix A /. W ef o r mthe c haracteristic equation// / / /// / / a/1/1 /; / a/1/2 /
/
/
a/1 n/0 a/2/2 /; / /
/
/
a/2 n/
/
/
/
/
/
/
/
/
/
/
/
/0 /0 /
/
/
ann /; / // / / /// / / /= /0 /:Expanding the determinan tw e get from here/( a/1/1 /; / /)/( a/2/2 /; / /) /
/
/
/( ann /; / /)/= /0 /: /2Problem /2/./5/./6 /. /Find eigen v alues and eigen v ectors of the matrix A /,wherea /) A /= /2/6/4 /1 /1 /1/0 /2 /1/0 /0 /2 /3/7/5 /;; b /) A /= /2/6/6/6/4 /1 /2 /4 /; /3/0 /1 /7 /7/0 /0 /3 /8/0 /0 /0 /; /2 /3/7/7/7/5 /:Prop osition /2/./5/./7/. The eigen v ectors of the matrix A corresp onding todi/eren t eigen v alues are linearly indep enden t/.Pr o of/. Let x/1 /;; x/2 /;; /
/
/
/;; xk b e the eigen v ectors of the matrix A corresp ond/-ing to the di/eren t eigen v alues //1 /;;//2 /;; /
/
/
/;;/k /(k /= /2 /: n/)/. W ew i l l s h o wt h a tthe system of these eigen v ectors is linearly indep enden t/. Av oiding complex/-it y w e shall go through the pro of in case k /= /2 /: Let us supp ose that thean tithesis is v alid/, i/.e/./, the v ector system f x/1 /;; x/2 g is linearly indep enden t/:/9 /( //1 /;;//2 /)/: //1 x/1 /+ //2 x/2 /=/0 /^ j //1 j /+ j //2 j /6/=/0 /: /(/1/1/)Multiplying the equalit y in /(/1/1/) on the left b y matrix A /,w e get//1 A x/1 /+ //2 A x/2 /=/0 /(/1/2/)or//1 //1 x/1 /+ //2 //2 x/2 /=/0 /: /(/1/3/)Multiplying the equalit y in /(/1/1/) b y //1 /, and substracting the result from /(/1/3/)/,w e get//2 /( //2 /; //1 /) x/2 /=/0 /:On the left in this equalit y only the /rst factor //2 can equal /0 /: Analogously /,m ultiplying in /(/1/1/) b y /(/1/1/) b y //2 /;; w e get the equalit y //1 /=/0 /. So j //1 j /+ j //2 j /=/5/2 /0 /;; and this is in con tradiction with the assumption /(/1/1/)/. Therefore/, thesystem of eigen v ectors f x/1 /;; x/2 g is linearly indep enden t/. /2Let us supp ose that the system of eigen v ectors f x/1 /;;/:/:/: /;; xn g of the matrixA is linearly indep enden t/. Let us form the n / n /; matrix S/;; c ho osing thev ector x/1 /;; as the /rst column/-v ector/, the v ector x/2 as the second column/-v ector/, /:/:/: /;; the v ector xn as the n/-th column/-v ector/, i/.e/./,S /= hx/1 /
/
/
xn i/: /(/1/4/)Let us denote//= /2/6/6/4 //1 /
/
/
/0/././. /
/
/
/./././0 /
/
/
/n /3/7/7/5 /: /(/1/5/)F or the ab o v e example /2/./5/./1/, w e getS /= /2/6/4 /; /1 /; /1 /1/0 /1 /1/1 /0 /1 /3/7/5 /^ //= /2/6/4 /0 /0 /0/0 /0 /0/0 /0 /3 /3/7/5 /:Prop osition /2/./5/./8/. If the matrix A has n linearly indep enden t eigen v ec/-tors x/1 /;; /
/
/
/;; xn corresp onding to the eigen v alues //1 /;; /
/
/
/;;/n /;; then the matrixA can be expressed in formA /= S / S /; /1/;; /(/1/6/)where the matrices S and / are de/ned b y /(/1/4/) and /(/1/5/)/.F or the pr o of it will su/ce to sho w thatAS /= S / /: /(/1/7/)Let us start from the left side of /(/1/7/)/:AS /= A hx/1 /
/
/
xn i/=/= hA x/1 /
/
/
A xn i/= h//1 x/1 /
/
/
/n xn i/:F rom the righ t s i d eo f/( /1 /7 /)w e get/:S //= hx/1 /
/
/
xn i /2/6/6/4 //1 /
/
/
/0/./. /. /
/
/
/./. /./0 /
/
/
/n /3/7/7/5 /=/5/3 /= h//1 x/1 /
/
/
/n xn i/:Therefore/, equalit y /(/1/7/) holds/, and consequen tly equalit y /(/1/6/)/, and also theequalit y/ /= S /; /1AS/: /2 /(/1/8/)Example /2/./5/./3 /. /Find a /3 / /3/-matrix A whose eigen v alues and corre/-sp onding eigen v actors are/://1 /=/3 /) x/1 /= h/; /3 /2 /1 iT/;;//2 /= /; /2 /) x/2 /= h/; /2 /1 /0 iT/;;//3 /=/1 /) x/3 /= h/; /6 /3 /1 iT/:As the w an ted matrix A can b e reprezen ted in form A /= S / S /; /1/;; whereS /= hx/1 x/2 x/3 i/^ //= /2/6/4 /3 /0 /0/0 /; /2 /0/0 /0 /1 /3/7/5 /;;thenA /= /2/6/4 /; /3 /; /2 /; /6/2 /1 /3/1 /0 /1 /3/7/5 /2/6/4 /3 /0 /0/0 /; /2 /0/0 /0 /1 /3/7/5 /2/6/4 /; /3 /; /2 /; /6/2 /1 /3/1 /0 /1 /3/7/5 /; /1/=/:/= /2/6/4 /; /9 /4 /; /6/6 /; /2 /3/3 /0 /1 /3/7/5 /2/6/4 /1 /2 /0/1 /3 /; /3/; /1 /; /2 /1 /3/7/5 /= /2/6/4 /1 /6 /; /1/8/1 /0 /9/2 /4 /1 /3/7/5 /:Problem /2/./5/./7 /. /Find a /2 / /2/-matrix A whose eigen v alues and corre/-sp onding eigen v ectors are/://1 /=/1 /) x/1 /= /"/3/4 /#/^ //2 /=/2 /) x/2 /= /"/5/7 /#/:Problem /2/./5/./8 /. /Find a /3 / /3/-matrix A whose eigen v alues and corre/-sp onding eigen v ectors are/://1 /=/3 /) x/1 /= h/; /1 /; /1 /1 iT/;;/5/4 //2 /= /; /3 /) x/2 /= h/2 /1 /0 iT/;;//3 /=/5 /) x/3 /= h/0 /0 /1 iT/:Example /2/./5/./4 /. /Find matrices A /1/0/0and A /1/5/5/;; whereA /= /"/4/1 /; /3/0/5/6 /; /4/1 /#/:Since/// / / /4/1 /; / /; /3/0/5/6 /; /4/1 /; / /// / / /=/0 /) / /2/; /1/= /0//1 /=/1 /) x/1 /= /"/3/4 /#/^ //2 /= /; /1 /) x/2 /= /"/5/7 /#/;;andA /= S / S /; /1/^ //= /"/1 /0/0 /; /1 /#/^ S /= /"/3 /5/4 /7 /#/^ S /; /1/= /"/7 /; /5/; /4 /3 /#/;;thenA /1/0/0/=/( S / S /; /1/)/( S / S /; /1/) /
/
/
/( S / S /; /1/)/= S / /1/0/0S /; /1/=/= /"/3 /5/4 /7 /#/"/1 /1/0/0/0/0 /( /; /1/) /1/0/0 /#/"/7 /; /5/; /4 /3 /#/= /"/1 /0/0 /1 /#/= IandA /1/5/5/= /"/3 /5/4 /7 /#/"/1 /1/5/5/0/0 /( /; /1/) /1/5/5 /#/"/7 /; /5/; /4 /3 /#/= /"/4/1 /; /3/0/5/6 /; /4/1 /#/= A/:Problem /2/./5/./9 /. /Find matrices A /1/0/0and A /1/5/5/;; wherea /) A /= /"/; /5 /2/; /2/1 /8 /#/;; b /) A /= /"/; /2/0 /4/2/; /9 /1/9 /#/:Prop osition /2/./5/./9/. If all the eigen v alues of the matrices A and B aresingle and the matrices A and B are comm utativ e/, then they ha v e commoneigen v ectors/.Pr o of/. Let x be an eigen v ector of the matrix A corresp onding to theeigen v alue / /, i/.e/./, it holds /(/6/)/. Let us m ultiply b oth sides /(/6/) on the left b y/5/5 the matrix B /. Due to the comm utabilit y of the matrices A and B /, w e getthe c hain/:A x /= /x /) B /( A x /)/= B /( / x /) /, /( BA /) x /= / /( B x /) /, A /( B x /)/= / /( B x /) /:Th us/, if x is an eigen v ector of the matrix A corresp onding to the eigen v alue/ /, then B x is also an eigen v ector of the matrix A corresp onding to the singleeigen v alue is a one/-dimensional subspace in R n/, then the v ectors x and B xare collinear/, i/.e/./,/9 / /: B x /= / x /:Th us/, the eigen v ector x of the matrix A corresp onding to the eigen v alue/ is also the eigen v ector of the matrix B corresp onding to the eigen v alue//: Analogously one can sho w that eac h eigen v ector of the matrix B is aneigen v ector of the matrix A /2Prop osition /2/./5/./1/0/. If the matrices A/;; B /2 C n / nha v e n commonlinearly indep enden t eigen v ectors/, then these matrices are comm utativ e/.Pr o of/. Due to prop osition /2/./5/./8/, these matrices can b e expressed in formA /= S / S /; /1/;; B /= S / S /; /1/;; /(/1/9/)where S is the matrix formed of the eigen v ectors as column/-v ectors/, and /is a diagonal matrix with eigen v alues of the matrix A on the main diagonal/,and / is a diagonal matrix with the eigen v alues of the matrix B on the maindiagonal/. Let us /nd the pro ducts AB and BA /;; using the represen tation in/(/1/9/)/:AB /= S / S /; /1S / S /; /1/= S // S /; /1andBA /= S / S /; /1S / S /; /1/= S // S /; /1/:As the diagonal matrices / and / are comm utativ e/, AB /= BA /;; q/.e/.d/. /2/1/./2/./6 Sc h ur/'s Decomp ositionThe eigen v ector x of the matrix A /2 C n / ndetermines in the space C naone/-dimensional subspace that is in v arian t with resp ect to the m ultiplicationb y the matrix A on the left/./5/6 De/nition /2/./6/./1/. The subspace S / C nis called in v arian t with resp ectto the m ultiplication b y the matrix A on the left if x /2 S /) A x /2 S/:Prop osition /2/./6/./1/. If A /2 C n / n/;; B /2 C k / k/;; X /2 C n / kand AX /=XB /;; then the space R /( X /) of the matrix X is in v arian t with resp ect to them ultiplication b y the matrix A on the left and the space R /( X T/)i si n v arian twith resp ect to the m ultiplication b y the matrix B on the righ t/. In addition/,the follo wing connectionsdim R /( X /)/= k /) / /( B /) / / /( A /)anddim R /( X /)/= k /= n /) / /( B /)/= / /( A /) /:holds/.Pr o of/. If X /=/[ c/1 /:/:/: ck /] /;; thenAX /= A /[ c/1 /:/:/: ck /]/= /[ A c/1 /:/:/: A ck /]andXB /=/[ c/1 /:/:/: ck /] /2/6/6/4 b/1/;;/1 /
/
/
b/1/;; k/././. /././.bk /;;/1 /:/:/: bk /;; k /3/7/7/5 /=/= hb/1/;;/1 c/1 /+ /:/:/: /+ bk /;;/1 ck /
/
/
b/1/;; k c/1 /+ /:/:/: /+ bk /;; k ck iandA ci /= b/1/;; i c/1 /+ /:/:/: /+ bk /;; i ck /(i /= /1 /: k /) /) A R /( X /) /R /( X /) /:Therefore/, the space R /( X /) of the column/-v ectors of the matrix X is in v arian twith resp ect to the m ultiplication b y the matrix A on the left/. Analogously /,one can pro v e that the space R /( X T/) of the ro w/-v ectors is in v arian t withresp ect to the m ultiplication b y the matrix B on the righ t/. If B y /= / y /;; thenA /( X y /)/= /( XB /) y /= X/ y /= / /( X y /) /;;i/.e/./, / is an eigen v alue of the matrix A if / is a eigen v alue of the matrix B/:Naturally /,y /6/= /0 dim R /( X /)/= k/) X y /6/= /0 /:/5/7 therefore/, if the column/-v ectors of the matrix X are linearly indep enden t/,then / /( B /) / / /( A /) /: If A z /= / z and X is a regular square matrix /(dim R /( X /)/=k /= n /)/, then it follo ws from the equalit y AX /= XB that A /= XB X /; /1andXB X /; /1z /= / z /, B /( X /; /1z /)/= / /( X /; /1z /) /;;i/.e/./, ev ery eigen v alue of the matrix A is an eigen v alue of the matrix B /, / /( A /) // /( B /)/, and th us / /( B /)/= / /( A /) /: /2De/nition /2/./6/./2/. Matrices A/;; B /2 C n / nare said to be similar if thereexists a regular matrix X /2 C n / nsuc h that A /= XB X /; /1/:Due to prop osition /2/./6/./1 /(the last assertion/)/, the sp ectrum of t w os i m i l a rmatrices are equal/. W e can get this result also directly/:det /( A /; /I /) /= det/( XB X /; /1/; X/ IX /; /1/)/=/= det/( X /( B /; /I /) X /; /1/) /= det/( X /) det/( B /; /I /) det /( X /; /1/) /:Problem /2/./6/./1/. /Are the matrices A and B similar ifa /) A /= /2/6/4 /1 i /0i /2 /; /1/0 i /1 /3/7/5 /^ B /= /2/6/4 /1/+ i /7 /2/0 /1 /9/0 /0 /2 /; i /3/7/5 /;;b /) A /= /2/6/4 /2 /5/+/2 /5 i /2/5 /1/0/0/2/5 /1/0/0 /2 /5/+/2 /5 i/2 /5/+/2 /5 i /2/5 /1/0/0 /3/7/5 /^ B /= /2/6/4 /1/0/0 /3/5 /+ /2/0 i /; /5/+/1 /5 i/9/5 /+ /1/5 i /7/6 /+ /1/6 i /; /4 /3/+/1 /2 i/4/0 /; /2/0 i /; /4 /3/+/1 /2 i /4/9 /+ /9 i /3/7/5 /?Prop osition /2/./6/./2/. If T /2 C n / nandT /= /"T/1/;; /1 T/1/;; /2/0 T/2/;; /2 /#pq /:p qthen / /( T /)/= / /( T/1/;; /1 /) /[ / /( T/2/;; /2 /) /:Pr o of/. If T x /= / x /;; i/.e/./, / /2 / /( T /) /;; x /= /"x/1x/2 /#/;; x/1 /2 C pand x/2 /2 C q/;;then/"T/1/;; /1 T/1/;; /2/0 T/2/;; /2 /#/"x/1x/2 /#/= / /"x/1x/2 /#/) /(T/1/;; /1 x/1 /+ T/1/;; /2 x/2 /= / x/1T/2/;; /2 x/2 /= / x/2 /:/5/8 If x/2 /6/= /0 /;; thenT/2/;; /2 x/2 /= / x/2 /) / /2 / /( T/2/;; /2 /) /:If x/2 /= /0 /;; thenT/1/;; /1 x/1 /= / x/1 /) / /2 / /( T/1/;; /1 /) /:Th us/,/ /( T /) / / /( T/1/;; /1 /) /[ / /( T/2/;; /2 /) /:Since the p otencies of the sets / /( T/1/;; /1 /) /[ / /( T/2/;; /2 /) and / /( T /) are equal/, thenthe prop osition holds/. /2Example /2/./6/./1/. Using prop osition /2/./6/./2/, let us /nd the sp ectrum of thematrixA /= /2/6/6 /6/4 /1 /1 /5 /6/; /1 /1 /7 /3/0 /0 /2 /1/0 /0 /; /4 /3 /3/7/7 /7/5 /:First/, w e /nd the eigen v alues of the matrices /"/1 /1/; /1 /1 /#and /"/2 /1/; /4 /3 /#/:// / // /1 /; / /1/; /1 /1 /; / // / // /=/0 /) /(//1 /=/1 /+ i//2 /=/1 /; i /;;//// / /2 /; / /1/; /4 /3 /; / //// / /=/0 /) /(//3 /=/( /5 /+ i p /1/5 /) /= /2//4 /=/( /5 /; i p /1/5 /) /= /2 /:Th us/, the sp ectrum of the matrix is / /( A /)/= f /1/+ i /;; /1 /; i /;; /(/5 /+ i p /1/5 /) /= /2/;; /(/5 /;i p /1/5/) /= /2 g /:Problem /2/./6/./2/. /Find b y the use of prop osition /2/./6/./2 the sp ectrum ofthe matrix A ifa /) A /= /2/6/6/6/4 /2 /; /3 /1/7 /3/6/4 /6 /1/1 /; /1/3/0 /0 /4 /4/0 /0 /3 /8 /3/7/7/7/5 /;; b /) A /= /2/6/4 /2 /1/7 /; /2/0 /; /2 /; /1/0 /5 /2 /3/7/5 /:De/nition /2/./6/./3/. A matrix Q /2 C n / nis called a unitary matrix ifQ HQ /= QQ H/= I/:/5/9 Problem /2/./6/./3/. /Is the matrix Q a unitary matrix ifa /) Q /= /"/; /1 /2 /1 /2 p /3/1 /2 p /3 /1 /2 /#/;; b /) Q /= /"cos x i sin xi sin x cos x /#/;;c /) Q /= /"/2 /5 p /5 /1 /5 i p /5/; /1 /5 p /5 /2 /5 i p /5 /#/:Prop osition /2/./6/./3 /( the QR factorisation theorem/) /. If A /2 C m / n/, thenthe matrix A can b e expressed in form A /= QR /;; where matrix Q /2 C m / misunitary matrix and matrix R /2 C m / nis an upp er triangular matrix/.Prop osition /2/./6/./4/. If A /2 C n / n/;; B /2 C p / p/;; X /2 C n / p/;;AX /= XB /(/2/0/)and r ank /( X /)/= p/;; then there exists a unitary matrix Q /2 C n / nsuc h thatQ HAQ /= T /= /"T/1/;; /1 T/1/;; /2/0 T/2/;; /2 /#pn /; p /;;p n /; pwhere / /( T/1/;; /1 /)/= / /( A /) /\ / /( B /) /:Pr o of/. Let us consider for the matrix X its QR factorization X /=Q /"R/1/0 /#/;; where Q /2 C n / nand R/1 /2 C p / p/: Substracting the factorizationof the matrix X in to equalit y /(/2/0/)/, w e getAQ /"R/1/0 /#/= Q /"R/1/0 /#B /, Q HAQ /"R/1/0 /#/= /"R/1/0 /#B /:The sp ectrums of the matrices Q HAQ and A coincide/, i/.e/. / /( Q HAQ /)/= / /( A /) /:Represen ting the matrix A in formQ HAQ /= /"T/1/;; /1 T/1/;; /2T/2/;; /1 T/2/;; /2 /#pn /; p /;;p n /; pw e /nd/"T/1/;; /1 T/1/;; /2T/2/;; /1 T/2/;; /2 /#/"R/1/0 /#/= /"R/1 B/0 /#/) /(T/1/;; /1 R/1 /= R/1 BT/2/;; /1 R/1 /=/0 prop/. /2/./6/./1/= /)det R/1 /6/=/0 /(/ /( T/1/;; /1 /)/= / /( B /)T/2/;; /1 /=/0 /: /:/6/0 Therefore/, the prop osition holds/. /2Remark /2/./6/./1/. Prop osition /2/./6/./4 mak es it p ossible/, if w e kno w an in/-v arian t subspace of the giv en matrix/, to transform it b y unitary similarit ytransformations in to a triangular blo c k form/.Prop osition /2/./6/./5 /(Sc h ur/'s decomp osition/)/. If A /2 C n / n/;; then thereexists a unitary matrix Q /2 C n / nsuc h thatQ HAQ /= T /= D /+ N/;; /(/2/1/)where D /= diag /( //1 /;; /:/:/: /;;/n /) and N /2 C n / nis a strictly upp er triangularmatrix/, i/.e/./, an upp er triangular matrix with zeros on the main diagonal/.The matrix Q can be formed so that the eigen v alues of the matrix A are inthe giv en order on the main diagonal of the matrix D /.T o pr ove this assertion w e will use the metho d of complete induction/. Asthe assertion holds for n /= /1/, the base for the induction exists/. W e are goingno w to sho w the admissibilit y of the induction steps/. W e supp ose that theassertion holds for the matrices whose order is less or equal to k /; /1 /: Let ussho w that the assertion will b e v alid for k /,t o o /. If A x /= / x and x /6/= /0 /;; then/,b y lemma /2/./6/./4/, c ho osing X /= x /;; B /= / /, there exists a unitary matrix U suc hthatU HAU /= T /= /"/ w H/0 C /#/1k /; /1 /;;/1 k /; /1Since C /2 C /( k /; /1/) / /( k /; /1/)/;; the assertion is v alid for this matrix/, i/.e/./, thereexists a unitary matrix /^U suc h that /^U HC /^U is an upp er triangular matrix/. IfQ /= U diag /(/1/;; /^U /) /;; thenQ HAQ /= /"/1 /0/0 /^U H /#U HAU /"/1 /0/0 /^U /#/=/= /"/1 /0/0 /^U H /#/"/ w H/0 C /#/"/1 /0/0 /^U /#/=/= /"/ w H/0 /^U HC /#/"/1 /0/0 /^U /#/= /"/ w H/^U/0 /^U HC /^U /#/;;and so the matrix Q HAQ is an upp er triangular matrix/. /2/6/1 Example /2/./6/./2/. LetA /= /"/3 /8/; /2 /3 /#and Q /= /"/2 i/= p /5 /1 /= p /5/; /1 /= p /5 /; /2 i/= p /5 /#/:Let us sho w that Q is unitary matrix/. F or this w e/, /rst/, /nd the pro ductQ HAQ/: The c hec king of the matrix Q for unitarit y giv es/:Q HQ /= /"/; /2 i/= p /5 /; /1 /= p /5/1 /= p /5 /2 i/= p /5 /#/"/2 i/= p /5 /1 /= p /5/; /1 /= p /5 /; /2 i/= p /5 /#/= /"/1 /0/0 /1 /#/;;QQ H/= /"/2 i/= p /5 /1 /= p /5/; /1 /= p /5 /; /2 i/= p /5 /#/"/; /2 i/= p /5 /; /1 /= p /5/1 /= p /5 /2 i/= p /5 /#/= /"/1 /0/0 /1 /#/:QQ H/= /"/2 i/= p /5 /1 /= p /5/; /1 /= p /5 /; /2 i/= p /5 /#/"/; /2 i/= p /5 /; /1 /= p /5/1 /= p /5 /2 i/= p /5 /#/= /"/1 /0/0 /1 /#/:Let us /nd the pro ductQ HAQ /= /"/; /2 i/= p /5 /; /1 /= p /5/1 /= p /5 /2 i/= p /5 /#/"/3 /8/; /2 /3 /#/"/2 i/= p /5 /1 /= p /5/; /1 /= p /5 /; /2 i/= p /5 /#/=/= /"/3/+/4 i /; /6/0 /3 /; /4 i /#/:Consequen tly /, w e ha v e obtained the Sc h ur decomp osition of the matrix A /.No w /(/2/1/) can be represen ted in the form AQ /= QT /: Replacing Q /=/[ q/1 /
/
/
qn /] /;; where the v ectors qi are called Schur ve ctors /,i n to the last equalit y/,w e getA /[ q/1 /
/
/
qn /]/= /[ q/1 /
/
/
qn /] Tor/[ A q/1 /
/
/
A qn /]/=/= h//1 q/1 //2 q/2 /+ n/1/;;/2 q/1 /
/
/
/n qn /+ n/1/;; n q/1 /+ n/2/;; n q/2 /+ /:/:/: /+ nn /; /1/;; n qn /; /1 iorA qi /= /i qi /+ n/1/;; i q/1 /+ /:/:/: /+ ni /; /1/;; i qi /; /1 /= /i qi /+ i /; /1Xk /=/1 nki qk /( i/= /1/: n/)/./6/2 F rom this equalit y it turns out that all subspaces Sk /= span f q/1 /;; /:/:/: /;; qk g /(k /= /1/: n/) are in v arian t with resp ect to m ultiplication b y the matrix A onthe left/, and Sc h ur v ector qi is an eigen v ector of the matrix A if and only ifin the i /-th column of the matrix N there are only zeros/.De/nition /2/./6/./4/. If A /2 C n / nand A HA /= AA H/;; then A is called anormal matrix/.Exercise /2/./6/./4/./* Chec k the normalit yo f A ifa /) A /= /2/6/4 /1 /; /1 /; /1/1 i /1/; /1 /; /1 /1 /3/7/5 /;; b /) A /= /2/6/4 i /; /1 i/1 i /1i /; /1 i /3/7/5 /;; c /) A /= /2/6/4 i i i/; i i /; ii i i /3/7/5 /:Prop osition /2/./6/./6/. A matrix A /2 C n / nis normal matrix i/ thereexists a unitary matrix Q /2 C n / n/;; satisfying the condition Q HAQ /= D /=diag /( //1 /;;/:/:/: /;;/n /) /:Pr o of/. If the matrix A is unitarily similar to the diagonal matrix D/;; thenQ HAQ /= D /, A /= QD Q H/) A HA /= QD HQ HQD Q H/= QD HDQ H/^AA H/= QD Q HQD HQ H/= QD D HQ Hand since diagonal matrices are comm utativ e/, then A HA /= AA Hand thematrix A is normal /. Vice v ersa/, if the matrix A is normal and the Sc h urdecomp osition of this matrix is Q HAQ /= T/;; then T is also normal b ecauseT HT /= Q HA HQQ HAQ /= Q HA HAQandTT H/= Q HAQQ HA HQ /= Q HAA HQ/:Since a triangular matrix is normal only if it is a diagonal matrix/, then ithas be e n pro v ed that a unitary matrix is similar to a diagonal matrix/. /2Prop osition /2/./6/./7 /(blo c k/-diagonal/-decomp osition/) /. LetQ HAQ /= T /= /2/6/6 /6/4 T/1/;; /1 T/1/;; /2 /
/
/
T/1/;; q/0 T/2/;;/2 /
/
/
T/2/;; q/
/
/
/
/
/
/
/
/
/
/
/
/0 /0 /
/
/
Tq /;; q /3/7/7 /7/5/6/3 b e the Sc h ur decomp osition of the matrix A /2 C n / n/, where the blo c ks Ti /;; i aresquare matrices/. If / /( Ti /;; i /) /\ / /( Tj /;; j /)/= /;; /( i /6/= j /) /;; then there exists a regularmatrixY /2 C n / nsuc h that/( QY /) /; /1A /( QY /)/= diag /( T/1/;;/1 /;; /:/:/: /;;Tq /;; q /) /:Corollary /2/./6/./1/. If A /2 C n / n/;; then there exists a regular matrix Xsuc h thatX /; /1AX /= diag /( //1 I /+ N/1 /;;/:/:/: /;; /q I /+ Nq /) Ni /2 C ni / ni/;;where //1 /;;/:/:/: /;;/q /, n/1 /+ /:/:/: /+ nq /= n and eac h Ni is a strictly upp er triangularmatrix/.Prop osition /2/./6/./8 /(Jordan decomp osition/)/. If A /2 C n / n/;; then there ex/-ists a regular matrix X /2 C n / nsuc h that X /; /1AX /= J /= diag /( J/1 /;; /:/:/: /;; Jt /) /;; wherem/1 /+ /:/:/: /+ mt /= n /,Ji /= /2/6/6/6/6 /6 /6 /6/6/4 /i /1 /0 /
/
/
/0/0 /i /1 /././. /./. /././. /. /././. /././. /././. /./. /././. /. /././. /././. /./././1/0 /
/
/
/
/
/
/0 /i /3/7/7/7/7 /7 /7 /7/7/5is an mi / mi Jordan blo c k/, and the matrix J is called the Jor dan normalform of the matrix A /.Pr o of/. See Lank aster /(/1/9/8/2/, p/. /1/4/3/)/.Example /2/./6/./3/. Using /"Maple/" /, w e /nd the Jordan decomp ositionsA /= XJ X /; /1of t w o matrices/:/2/6/6/6 /6 /6 /6/4 /0 /0 /1 /0 /0/0 /0 /0 /1 /0/0 /0 /0 /0 /1/0 /0 /0 /0 /0/0 /0 /0 /0 /0 /3/7/7/7 /7 /7 /7/5 /= /2/6/6/6 /6 /6 /6/4 /1 /0 /1 /0 /0/0 /0 /0 /1 /0/0 /1 /0 /0 /0/0 /0 /0 /0 /1/0 /0 /1 /0 /0 /3/7/7/7 /7 /7 /7/5 /2/6/6/6 /6 /6 /6/4 /0 /1 /0 /0 /0/0 /0 /1 /0 /0/0 /0 /0 /0 /0/0 /0 /0 /0 /1/0 /0 /0 /0 /0 /3/7/7/7 /7 /7 /7/5 /2/6/6/6 /6 /6 /6/4 /1 /0 /0 /0 /; /1/0 /0 /1 /0 /0/0 /0 /0 /0 /1/0 /1 /0 /0 /0/0 /0 /0 /1 /0 /3/7/7/7 /7 /7 /7/5and/2/6/6/6/4 /1 /; /1 /0 /; /1/0 /2 /0 /1/; /2 /1 /; /1 /1/2 /; /1 /2 /0 /3/7/7/7/5 /= /2/6/6/6/4 /; /1 /2 /; /1 /3 /2 /; /1/1 /2 /1 /; /1 /2 /0/3 /2 /1 /; /3 /2 /1/; /3 /2 /; /1 /3 /2 /0 /3/7/7/7/5 /2/6/6/6/4 /; /1 /0 /0 /0/0 /1 /1 /0/0 /0 /1 /0/0 /0 /0 /1 /3/7/7/7/5 /2/6/6/6/4 /1 /0 /1 /0/0 /3 /2 /0 /1 /2/1 /1 /1 /1/0 /0 /1 /1 /3/7/7/7/5 /:/6/4 /1/./2/./7 Norms and Condition Num b ers of a MatrixDe/nition /2/./7/./1/. A mapping f /: R m / n/! R is called the norming ofa matrix and the obtained v alue the matrix norm if the follo wing threeconditions are satis/ed/:f /( A /) / /0 A /2 R m / n/;; /( f /( A /)/= /0 /, A /=/0 /)f /( A /+ B /) / f /( A /)/+ f /( B /) A/;; B /2 R m / n/;;f /( /A /)/= j / j f /( A /) / /2 R/;; A /2 R m / n/:The matrix norm will b e denoted f /( A /)/= k A k /:The most frequen tly used norms in linear algebra are the F r ob enius normk A kF /= vuu t mXi /=/1 nXj /=/1 j aij j /2/(/2/2/)and the p /; norms /( p / /1/) /;;k A kp /= supx /6/= /0 k A x kp k x kp /: /(/2/3/)F rom /(/2/3/) it follo ws thatk A kp x /6/= /0/ k A x kp /= k x kpork A x kp x /6/= /0/ k A kp k x kp /(/2/4/)Let us v erify that p /; norm satis/es the conditions of the matrix norm/. W e/nd thatk A x kp / /0 /^ k x kp /> /0 /)k A x kp /= k x kp / /0 /)k A kp /=s u px /6/= /0 k A x kp /= k x kp / /0 /;;k A kp /= supx /6/= /0 k A x kp /= k x kp /=/0 /,k A x kp /=/0 /8 x /2 R n/, A /=/0 /;;furtherk A /+ B kp /= supx /6/= /0 k /( A /+ B /) x kp /= k x kp /= supx /6/= /0 k A x /+ B x kp /= k x kp //6/5 / supx /6/= /0 /( k A x kp /+ k B x kp /) /= k x kp / supx /6/= /0 k A x kp /= k x kp /+ supx /6/= /0 k B x kp /= k x kp /=/= k A kp /+ k B kpandk /A kp /=s u px /6/= /0 k /( /A /) x kp /= k x kp /= supx /6/= /0 j / jk A x kp /= k x kp /=/= j / j supx /6/= /0 k A x kp /= k x kp /= j / jk A kp /:Exercise /2/./7/./1/. V erify that the F rob enius norm satis/es the conditionsof the matrix norm/.Exercise /2/./7/./2/./* Compute the F rob enius norm k A kF ifa /) A /= /2/6/4 /1 /2 /3/0 /5 /4/2 /1 /3 /3/7/5 /;; b /) A /= /2/6/6 /6/4 /0 /0 /1 /2/3 /0 /5 /4/1 /1 /1 /2/1 /3 /2 /2 /3/7/7 /7/5 /;; c /) A /= /2/6/6 /6 /6/6/6/4 /1 /1 /1 /1 /1/2 /3 /4 /5 /6/0 /1 /0 /1 /0/3 /4 /3 /4 /3/5 /5 /5 /5 /5 /3/7/7 /7 /7/7/7/5 /:De/nition /2/./7/./2/. F or the /xed matrix norm the v aluek /( A /) /= k A k / / / A /; /1 / / / is called the c ondition numb er corresp onding to the regular square matrixA /2 R n / n/:The condition n um be r corresp onding to the F rob enius norm will be de/-noted kF /( A /) and the condition n um be r corresp onding to the p /; norm willbe denoted kp /( A /) /: F or a singular square matrix A /2 R n / nw e will de/nek /( A /)/= /+ /1 /:Exercise /2/./7/./2/. Sho w that if A /2 R n / n/;; then/1 n k/2 /( A /) / k/1 /( A /) / nk/2 /( A /) /;;/1 n k/1 /( A /) / k/2 /( A /) / nk/1 /( A /) /;;/1 n /2 k/1 /( A /) / k/1 /( A /) / n /2k/1 /( A /) /:/6/6 Prop osition /2/./7/./1/. Rule /(/2/3/) for the calculation of the norm k A kp canbe transformed to the formk A kp /= supk x kp /=/1 k Ax kp /: /(/2/5/)Pr o of/. Using the third prop ert y of the norm and the homogeneit y ofm ultiplication of a v ector b y a matrix/, w e ha v ek Ax kp k x kp /= / / / / / /1 k x kp Ax / / / / / p /= / / / / / A x k x kp / / / / / p /;;where / / / / x k x kp / / / / p /=/1 /: /2Prop osition /2/./7/./2/. If A /2 R m / n/, B /2 R n / qand p / /1 /;; then k AB kp /k A kp k B kp /:Pr o of/. Using /(/2/4/) and /(/2/5/)/, w e /nd thatk AB kp /= supk x kp /=/1 k /( AB /) x kp /= supk x kp /=/1 k A /( Bx /) kp / supk x kp /=/1 k A kp k Bx kp /=/= k A kp supk x kp /=/1 k Bx kp /= k A kp k B kp /: /2Remark /2/./7/./1/. Since kp /( A /) /= k A kp k A /; /1kp / k AA /; /1kp /= k I kp /= /1 /;;then alw a ys kp /( A /) / /1/.Remark /2/./7/./2/. F or eac h A /2 R m / nand x /2 R nand for arbitraryv ector norm k/
k/ on R nand k/
k/ on R mthe relationk A x k/ /k A k//;;/ k x k/ /;;holds/, where k A k//;;/ is a matrix norm de/ned b yk A k//;;/ /= supx /6/= /0 k A x k/ /= k x k/ /:Since the set f x /2 R n/: k x k/ /= /1 g is compact and k/
k/ is con tin uous/, itfollo ws thatk A k//;;/ /= maxk x k/ /=/1 k A x k/ /= k A x /k/for some x //2 R nwith k x /k/ /=/1 /:/6/7 De/niton /2/./7/./3/. If k /( A /) is relativly small/, then the matrix A is called awel l/-c onditione d matrix /, but if k /( A /) is great/, then an il l/-c onditione d matrix /.De/nition /2/./7/./4/. A norm k A k of the square matrix A is said to bec onsistent with the v ector norm k x k ifk A x k /k A k k x kand it is subm ultiplicativ e/, i/.e/./,k AB k/ k A k k B k /:De/nition /2/./7/./5/. The norm k A k of the square matrix/, consisten t withthe v ector norm k x k is said to be sub or dinate to the ve ctor norm k x k iffor an y matrix A there exists a v ector x /= x /( A /) /6/= /0 suc h that k A x k /=k A kk x k /:Prop osition /2/./7/./3/. F or arbitrary v ector norm k x k there exists at leastone matrix norm k A k sub ordinate /(and th us at least one consisten t /) to thisv ector norm/, and this isk A k /= maxk x k /=/1 k A x k /=maxx /6/= /0 k A x k k x k /:Remark /2/./7/./3/. Not all matrix norms satisfy the subm ultiplicativ ep r o p /-ert y k AB k / k A kk B k /. F or example/, if w e de/ne k A k/
/= maxi/;; j j aij j /, thenfor the matrices A /= B /= /"/1 /1/0 /1 /#w e ha v e k A k/
/= k B k/
/= /1 andk AB k/
/= jj /"/1 /2/0 /1 /#jj/
/=/2 /> k A k/
k B k/
/:Prop osition /2/./7/./4/. If A /2 R m / n/, then the follo wing relations be t w eenthe matrix norms hold/:k A k/1 /= max/1 / j / n mXi /=/1 j aij j /;; /(/2/6/)k A k/1 /= max/1 / i / m nXj /=/1 j aij j /;; /(/2/7/)/6/8 k A k/2 /k A kF / p n k A k/2 /;;k A k/
/k A k/2 / p mn k A k/
/;;/1 p n k A k/1 /k A k/2 / p m k A k/1 /;;/1 p n k A k/1 /k A k/2 / p n k A k/1 /:If A /2 C m / n/;; /1 / i/1 / i/2 / m and /1 / j/1 / j/2 / n/;; thenk A /( i/1 /: i/2 /, j/1 /: j/2 /) kp /k A kp /:Pr ove the relation /(/2/7/) /. W e ha v ek Ax k/1 /= maxi //// / / nXj /=/1 aij /j //// / / / maxi nXj /=/1 j aij jj /j j// maxi nXj /=/1 j aij jk x k/1 /= k x k/1 nXj /=/1 j akj j /;;where w e supp ose that maxim um has b een gained if the index i obtains thev alue k /. W e ha v e the estimationk A k/1 / maxi nXj /=/1 j aij j /:Letz /= //&/1 /:/:/: /&n /Tand/&j /= /(/1 /;; kui akj / /0/;;/; /1 /;; kui akj /< /0 /:Since k z k/1 /=/1 /;; thenk A k/1 /= supk x k/1 /=/1 k Ax kp /k Az k/1 /= maxi // /// / nXj /=/1 aij /&j // /// / / nXj /=/1 akj /&k /= nXj /=/1 j akj j /;;and th usk A k/1 /= maxi nXj /=/1 j aij j /: /2/6/9 Example /2/./7/./1/. Let us calculate the norms k A k/1 and k A k/1 for thematrix A ifA /= /2/6/4 a/1/1 a/1/2 a/1/3a/2/1 a/2/2 a/2/3a/3/1 a/3/2 a/3/3 /3/7/5 /:F rom /(/2/6/) and /(/2/7/) w e /nd that/ / / / / / / a/1/1 a/1/2 a/1/3a/2/1 a/2/2 a/2/3a/3/1 a/3/2 a/3/3 / / / / / / / /1 /=/= max /( j a/1/1 j /+ j a/2/1 j /+ j a/3/1 j /;; j a/1/2 j /+ j a/2/2 j /+ j a/3/2 j /;; j a/1/3 j /+ j a/2/3 j /+ j a/3/3 j /)and/ / / / / / / a/1/1 a/1/2 a/1/3a/2/1 a/2/2 a/2/3a/3/1 a/3/2 a/3/3 / / / / / / / /1 /=/= max /( j a/1/1 j /+ j a/1/2 j /+ j a/1/3 j /;; j a/3/1 j /+ j a/3/2 j /+ j a/3/3 j /;; j a/2/1 j /+ j a/2/2 j /+ j a/2/3 j /) /:Example /2/./7/./2/. Let us calculate the in v erse matrix A /; /1of the matrix A /,the norms k A k/1 /;; k A /; /1k/1 /;; k A k/2 /;; k A /; /1k/2 /;; k A k/1 /;; k A /; /1k/1 and the conditionn um b ers of matrix A k/1 /( A /)/, k/2 /( A /) /;;k/1 /( A /) ifA /= /2/6/4 /; /2 /1 /0/1 /; /2 /1/0 /1 /; /2 /3/7/5 /:It follo ws thatA /; /1/= /2/6/4 /; /3 /4 /; /1 /2 /; /1 /4/; /1 /2 /; /1 /; /1 /2/; /1 /4 /; /1 /2 /; /3 /4 /3/7/5 /;;k A k/1 /= k A k/1 /=/4 /;; k A k/2 /=/2 /+ p /2 /;; / / / A /; /1 / / / /1 /= / / / A /; /1 / / / /1 /=/2and/ / / A /; /1 / / / /2 /=/1 /+ /1 /2 p /2 /;; k/1 /( A /)/= k/1 /( A /)/= /2 /;; k/2 /( A /)/= /1 /2 //2/+ p /2 //2/:If form ulae /(/2/6/) and /(/2/7/) enable us to calculate easily /1 /; norm and /1/; norm/,resp ectiv ely /, then the calculation of the /2 /; norm is more complicated/. Thematrix /2 /; norm is called also the matrix sp e ctr al norm/./7/0 Prop osition /2/./7/./5/. If A /2 R m / n/;; thenk A k/2 /= r max/ /2 / /( A TA /) / /;;i/.e/./, k A k/2 is the square ro ot of the largest eigen v alue of A TA /.Pr o of/. T o calculate k A k/2 /;; w e /nd /rst k A k /2/2 /: Th us/,k A k/2 /= maxk x k/2 /=/1 k A x k/2 /) k A k /2/2 /= maxk x k/2 /=/1 k A x k /2/2 /=m a xx Tx /=/1 x TA TA x /:Let A TA /= B /2 R n / n/: The matrix B is a symmetric matrix b ecauseB T/=/( A TA /) T/= A TAandx TA TA x /= x TB x /= h//1 /
/
/
/n i /2/6/6/4 b/1/;; /1 /
/
/
b/1/;; n/././. /
/
/
/././.bn /;;/1 /
/
/
bn /;; n /3/7/7/5 /2/6/6/4 //1/./././n /3/7/7/5 /=/= h//1 /
/
/
/n i /2/6/4 b/1/;; /1 //1 /+ /:/:/: /+ b/1/;; n /n/
/
/
/
/
/
/
/
/
bn /;;/1 //1 /+ /:/:/: /+ bn /;; n /n /3/7/5 /=/= /2/4//1 nXj /=/1 b/1/;; j /j /+ /:/:/: /+ /n nXj /=/1 bn /;; j /j /3/5/;;x Tx /= nXj /=/1 / /2j /;;then x TA TA x is a function of n v ariables //1 /;; /:/:/: /;;/n and/@ /( x TA TA x /) /@/i /= nXj /=/1 bi /;; j /j /+ nXj /=/1 bj /;; i /j bi /;; j /= bj /;; i/= /2 nXj /=/1 bi /;; j /j /=/2 hA TA x ii /;;/@ /( x Tx /) /@/i /=/2 /i /=/2 /[ x /]i /:The problem of /nding maxx Tx /=/1 x TA TA x is a problem of /nding the relativ eextrem um/. T o solv e our problem w e form an auxiliary function//( //1 /;; /:/:/: /;;/n /;; / /)/= x TA TA x /+ / /(/1 /; x Tx /) /:/7/1 T o /nd the stationary p oin ts of / /;; w e form the system of equations/:/@ / /@/i /=/0 /( i/= /1/: n/) /^ /@ / /@/ /=/0 /;;i/.e/./,/(/2 hA TA x ii /; /2 / /[ x /]i /=/0 /( i/= /1 /: n/)/1 /; x Tx /= /0or/(A TA x /= / x /;;k x k/2 /=/1 /:Th us/, an y stationary p oin t for relativ e extrem um is the normed v ectorx corresp onding to an eigen v alue of A TA /. Let us express from the relationA TA x /= / x the eigen v alue //: W e obtain that / /= x TA TA x /, where k x k/2 /=/1 /:Comparing this result with the original form ula for /nding k A k /2/2 /;; w e noticethat k A k /2/2 /= max/ /2 / /( A TA /) / /. Th us/,k A k/2 /= r max/ /2 / /( A TA /) //;;i/.e/./, k A k/2 is the square ro ot of the largest eigen v alue of A TA /.Corollary /2/./7/./1/. If matrix A /2 R m / nis symmetric/, thenk A k/2 /= max/ /2 / /( A /) //:Example /2/./7/./3/. Let us calculate the in v erse matrix A /; /1of the matrix A /,the norms k A k/1 /;; k A /; /1k/1 /;; k A k/2 /;; k A /; /1k/2 /;; k A k/1 /;; k A /; /1k/1 and the conditionn um b ers of the matrix Ak/1 /( A /)/, k/2 /( A /) /;; k/1 /( A /) ifA /= /"/1 /1/1 /1 /: /0/0/0/0/0/0/0/1 /#/:W e obtain thatA /; /1/= /"/1 /: /0 / /1/0 /8/; /1 /: /0 / /1/0 /8/; /1 /: /0 / /1/0 /8/1 /: /0 / /1/0 /8 /#/;; k A k/1 /= k A k/2 /k A k/1 / /2 /;;/ / / A /; /1 / / / /1 /= / / / A /; /1 / / / /1 / / / / A /; /1 / / / /2 / /2 / /1/0 /8/;; k/1 /( A /)/= k/1 /( A /) / /4 / /1/0 /8/:/7/2 Example /2/./7/./4/. Let us see ho w the almost singularit y /(the v alue of thedeterminan ti s close to zero/) and ill condition of the matrix are related/. F orthe matrixAn /= /2/6/6/6 /6 /6 /6/4 /1 /; /1 /; /1 /
/
/
/; /1/0 /1 /; /1 /
/
/
/; /1/0 /0 /1 /
/
/
/; /1/
/
/
/
/
/
/
/
/
/
/
/
/
/
/
/0 /0 /0 /
/
/
/1 /3/7/7/7 /7 /7 /7/5 /2 R n / ndet/( An /)/= /1 but k/1 /( An /)/= n /2 n /; /1/: In con trast/, for the diagonal matrixDn /= diag /( /" /;; /:/:/: /;;/" /) /2 R n / nkp /( Dn /)/= /1 but det /( Dn /)/= /" nfor an arbitrarily small /" /.Exercise /2/./7/./4/./* Find the in v erse A /; /1of the matrix A /, the normsk A k/1 /;; k A /; /1k/1 /;; k A k/2 /;; k A /; /1k/2 /;; k A k/1 /;; k A /; /1k/1 and the condition n um be r sk/1 /( A /)/, k/2 /( A /) /;; k/1 /( A /) ifa /) A /= /2/6/4 /0 /0 /1/0 /1 /0/1 /1 /1 /3/7/5 /;; b /) A /= /2/6/4 /3 /0 /0/0 /2 /0/0 /0 /1 /3/7/5 /;; c /) A /= /2/6/6 /6/4 /; /2 /; /1 /2 /; /1/1 /2 /1 /; /2/2 /; /1 /2 /1/0 /2 /0 /1 /3/7/7 /7/5 /:/1/./2/./8 Ca yley/-Hamilton TheoremProp osition /2/./8/./1 /(Ca yley/-Hamilton theorem/)/. If A /2 C n / nandp /( / /) /= det /( A /; /I /) /;;then p /( A /)/= /0 /, i/.e/./, the matrix A satis/es its c haracteristic equation /.Pr o of/. According to prop osition /2/./6/./7/, there exists a regular matrix X /2C n / nsuc h thatX /; /1AX /= J / diag /( J/1 /;; /:/:/: /;; Jt /) /;;whereJi /= /2/6/6 /6 /6/6/6/6/6/4 /i /1 /0 /
/
/
/0/0 /i /1 /././. /./. /././. /. /././. /././. /././. /./. /././. /. /././. /././. /./././1/0 /
/
/
/
/
/
/0 /i /3/7/7 /7 /7/7/7/7/7/5/7/3 is an upp er bidiagonal mi / mi /; matrix /(Jordan blo c k/) that has on its maindiagonal the eigen v alue /i of the matrix A /( at least mi /; m utiple eigen v alueof the matrix A since to this eigen v alue ma y corresp ond some more Jordanblo c ks/) and m/1 /+ /:/:/: /+ mt /= n/: Since Ji /; /i I /=/( /k /;; j /; /1 /) /;; then /( /k /;; j /; /1 /) mi/=/0and /( Ji /; /i I /) mi/=/0 /: If p /( / /) is the c haracteristic p olynomial of the matrixA and the zeros of this p olynomial are //1 /;; /:/:/: /;;/t /, thenp /( / /)/= /( /; /1/) n/( / /; //1 /) m/1/
/
/
/( / /; /t /) mtandp /( J /)/=/( /; /1/) n/( J /; //1 I /) m/1/
/
/
/( J /; /t I /) mt/:W e sho w that p /( J /)/= /0 /: Let the matrix J ha v e the blo c k form/:J /= /2/6/6/6 /6 /6 /6/6/4 J/1 /0 /0 /
/
/
/0/0 J/2 /0 /
/
/
/0/0 /0 J/3 /
/
/
/0/./. /. /./. /. /./. /. /././. /./. /./0 /0 /0 /
/
/
Jt /3/7/7/7 /7 /7 /7/7/5 m/1m/2m/3/./. /.mt /:W e obtain thatp /( J /)/= /( /; /1/) n/( J /; //1 I /) m/1/:/:/: /( J /; /t I /) mt/=/=/( /; /1/) n /2/6/6 /6 /6 /6/6/6/4 J/1 /; //1 I /0 /0 /
/
/
/0/0 J/2 /; //1 I /0 /
/
/
/0/0 /0 J/3 /; //1 I /./././0/././. /././. /././. /././. /./././0 /0 /0 /
/
/
Jt /; //1 I /3/7/7 /7 /7 /7/7/7/5 m/1/
/
/
/
/
/
/2/6/6/6/6 /6 /6 /6/4 J/1 /; /t I /0 /0 /
/
/
/0/0 J/2 /; /t I /0 /
/
/
/0/0 /0 J/3 /; /t I /./././0/././. /././. /././. /././. /./././0 /0 /0 /
/
/
Jt /; /t I /3/7/7/7/7 /7 /7 /7/5 mt/=/=/( /; /1/) n /2/6/6/6/6/6 /6 /6/4 /( J/1 /; //1 I /) m/1/0 /0 /
/
/
/0/0 /( J/2 /; //1 I /) m/1/0 /
/
/
/0/0 /0 /( J/3 /; //1 I /) m/1 /./././0/./. /. /./. /. /././. /././. /./. /./0 /0 /0 /
/
/
/( Jt /; //1 I /) m/1 /3/7/7/7/7/7 /7 /7/5 /
/
/
/7/4 /
/
/
/2/6/6/6 /6 /6 /6/6/4 /( J/1 /; /t I /) mt/0 /0 /
/
/
/0/0 /( J/2 /; /t I /) mt/0 /
/
/
/0/0 /0 /( J/3 /; /t I /) mt /./././0/./. /. /./. /. /././. /././. /./. /./0 /0 /0 /
/
/
/( Jt /; /t I /) mt /3/7/7/7 /7 /7 /7/7/5 /=/=/( /; /1/) n /2/6/6 /6/6/6/6 /6/4 /0 /0 /0 /
/
/
/0/0 /( J/2 /; //1 I /) m/1/0 /
/
/
/0/0 /0 /( J/3 /; //1 I /) m/1 /./././0/././. /././. /././. /././. /./././0 /0 /0 /
/
/
/( Jt /; //1 I /) m/1 /3/7/7 /7/7/7/7 /7/5 /
/
/
/
/
/
/2/6/6/6 /6 /6/6/6/4 /( J/1 /; /t I /) mt/0 /0 /
/
/
/0/0 /( J/2 /; /t I /) mt/0 /
/
/
/0/0 /0 /( J/3 /; /t I /) mt /./././0/./. /. /./. /. /././. /././. /./. /./0 /0 /0 /
/
/
/0 /3/7/7/7 /7 /7/7/7/5 /=/=/( /; /1/) n /2/6/6 /6/6/6 /6 /6/4 /0 /0 /0 /
/
/
/0/0 /0 /0 /
/
/
/0/0 /0 /0 /./././0/./. /. /./. /. /././. /././. /./. /./0 /0 /0 /
/
/
/0 /3/7/7 /7/7/7 /7 /7/5 /=/0 /:F rom the relation X /; /1AX /= J it follo ws that A /= XJ X /; /1/: W e completethe pro of withp /( A /)/= p /( XJ X /; /1/)/=/=/( /; /1/) n/( XJ X /; /1/; X//1 IX /; /1/)/( XJ X /; /1/; X//2 IX /; /1/) /
/
/
/( XJ X /; /1/; X/n IX /; /1/)/=/=/( /; /1/) nX /( J /; //1 I /) X /; /1X /( J /; //2 I /) X /; /1/
/
/
X /( J /; /n I /) X /; /1/= Xp /( J /) X /; /1/=/0 /: /2Example /2/./8/./1/. V erify the assertion of the Ca yley/-Hamilton theoremfor the matrixA /= /"a bc d /#/./7/5 W e construct the c haracteristic p olynomialp /( / /) /= det/( A /; /I /)/= // / / / a /; / bc d /; / // / / / /= / /2/; /( a /+ d /) / /+ ad /; cband /ndp /( A /)/= /"a bc d /#/2/; /( a /+ d /) /"a bc d /#/+/( ad /; bc /) /"/1 /0/0 /1 /#/=/= /"a /2/+ bc /; a /2/; ad /; bc ab /+ bd /; ab /; bdac /+ cd /; ac /; cd bc /+ d /2/; ad /; d /2/+ ad /; bc /#/=/0 /:Exercise /2/./8/./1/./* LetA /= /2/6/4 /2 /3 /1/3 /1 /2/1 /2 /3 /3/7/5 /:Compute A /2/;; and using the Ca yley/-Hamilton theorem/, /nd the matrixA /7/; /3 A /6/+ A /4/+/3 A /3/; /2 A /2/+/3 I/:De/nition /2/./8/./1/. A p olynomial q /( / /) is called a nul lifying p olynomial ofthe matrix A /2 C n / nif q /( A /)/= /0 /:The c haracteristic p olynomial of the matrix A /2 C n / nis a n ullifyingp olynomial of this matrix /(b y the Ca yley/-Hamilton theorem/)/.De/nition /2/./8/./2/. The n ullifying p olynomial of the matrix A /2 C n / nofthe lo w est degree is called the minimal p olynomial of the matrix A /.Exercise /2/./8/./1/. V erify that the c haracteristic p olynomial of matrixA /2 C n / nis divisible b y the minimal p olynomial of the matrix A withoutremainder/.Prop osition /2/./8/./2/. Let p /( / /) and / /( / /)b et h ec haracteristic p olynomialand the minimal p olynomial of the matrix A/;; resp ectiv ely /. Let the greatestcommon divisor of the matrix /( I/ /; A /) /_/;; that is/, the matrix of algebraiccomplemen ts of the elemen ts of the matrix /( I/ /; A /) /, be d /( / /)/. Then/,p /( / /)/= d /( / /) / /( / /) /:/7/6 Pr o of/. See Lank aster /(/1/9/8/2/, p/. /1/2/3/-/1/2/4/)/.Example /2/./8/./2/. Find the c haracteristic p olynomial and the minimalp olynomial of the matrix D /= diag /( a/;; a/;; b/;; b /) /: First w e /nd that/( I/ /; D /) /_/= diag /( / /; a/;; / /; a/;; / /; b/;; / /; b /) /_/=/= /2/6/6/6/4 / /; a /0 /0 /0/0 / /; a /0 /0/0 /0 / /; b /0/0 /0 /0 / /; b /3/7/7/7/5 /_/=/= /2/6/6 /6/4 /( / /; a /)/( / /; b /) /2/0 /0 /0/0 /( / /; a /)/( / /; b /) /2/0 /0/0 /0 /( / /; a /) /2/( / /; b /) /0/0 /0 /0 /( / /; a /) /2/( / /; b /) /3/7/7 /7/5and the greatest common divisor of the elemen ts of the matrix /( I/ /; D /) /_is d /( / /)/= /( / /; a /)/( / /; b /) /: By the assertion of prop osition /2/./8/./2 / /( / /) /=/( / /;a /)/( / /; b /) /: Let us c hec k/ /( D /)/= /( D /; aI /)/( D /; bI /)/=/= /2/6/6 /6/4 /0 /0 /0 /0/0 /0 /0 /0/0 /0 b /; a /0/0 /0 /0 b /; a /3/7/7 /7/5 /2/6/6 /6/4 a /; b /0 /0 /0/0 a /; b /0 /0/0 /0 /0 /0/0 /0 /0 /0 /3/7/7 /7/5 /=/0 /:Indeed/, / /( / /) is the n ullifying p olynomial/. It is easy to v erify that no p oly/-nomial of /rst degree can n ullify the matrix A/: Th us/, / /( / /) is the minimalp olynomial of the matrix A /.Example /2/./8/./3/. Find the c haracteristic and the minimal p olynomials ofthe matricesA /= /2/6/4 /6 /2 /; /2/; /2 /2 /2/2 /2 /2 /3/7/5 and B /= /2/6/4 /6 /2 /2/; /2 /2 /0/0 /0 /2 /3/7/5First w e /nd the c haracteristic p olynomials/:pA /( / /)/= // /// / / /6 /; / /2 /; /2/; /2 /2 /; / /2/2 /2 /2 /; / // /// / / /= / /3/; /1/0 / /2/+/3 /2 / /; /3/2/7/7 andpB /( / /)/= // / / / // /6 /; / /2 /2/; /2 /2 /; / /0/0 /0 /2 /; / // / / / // /= / /3/; /1/0 / /2/+/3 /2 / /; /3/2 /:Next w e /nd the minimal p olynomials/:/( I/ /; A /) /_/=/= /2/6/4 / /; /6 /; /2 /2/2 / /; /2 /; /2/; /2 /; /2 / /; /2 /3/7/5 /_/= /2/6/4 / /2/; /4 / /; /2 / /+/8 /2 / /; /8/2 / /; /8 / /2/; /8 / /+/1 /6 /2 / /; /8/; /2 / /+/8 /2 / /; /8 / /2/; /8 / /+/1 /6 /3/7/5 /=/= /2/6/4 / /( / /; /4/) /; /2/( / /; /4/) /2/( / /; /4/)/2/( / /; /4/) /( / /; /4/) /2/2/( / /; /4/)/; /2/( / /; /4/) /2/( / /; /4/) /( / /; /4/) /2 /3/7/5 /) dA /( / /)/= / /; /4 /)/) / A /( / /)/= / /3/; /1/0 / /2/+/3 /2 / /; /3/2 / /; /4 /= / /2/; /6 / /+/8and/( I/ /; B /) /_/=/= /2/6/4 / /; /6 /; /2 /; /2/2 / /; /2 /0/0 /0 / /; /2 /3/7/5 /_/= /2/6/4 /( / /; /2/) /2/; /2/( / /; /2/) /0/2/( / /; /2/) /( / /; /2/)/( / /; /6/) /0/2/( / /; /2/) /; /4 /( / /; /4/) /2 /3/7/5 /)/) dB /( / /)/= /1 /) / B /( / /)/= / /3/; /1/0 / /2/+/3 /2 / /; /3/2 /1 /= / /3/; /1/0 / /2/+/3 /2 / /; /3/2 /:/1/./2/./9 F unctions of MatricesLet us consider a matrix A /2 C n / nand a function of a complex v ariablef /( z /) /;; f /: C /! C /:There are a lot of p ossibilities to de/ne a function of matrix f /( A /) startingfrom the function of a complex v ariable f /( z /)/. The simplest of these p ossibil/-ities seems to be the substituting the v ariable /" z /" b y the v ariable /" A /" /: F orexample/,f /( z /)/= z /2/+/3 z /; /7 /! f /( A /)/= A /2/+/3 A /; /7 I/7/8 andf /( z /)/= /4/+/5 z /3 /; /8 z /! f /( A /)/= /( /4 I /+/5 A /)/(/3 I /; /8 A /) /; /1/;;as w elle z/= /1Xk /=/0 z k k /! /! e A/= /1Xk /=/0 A k k /! /;;cos z /= /1Xk /=/0 /( /; /1/) k z /2 k /(/2 k /)/! /! cos A /= /1Xk /=/0 /( /; /1/) k A /2 k /(/2 k /)/! /;;sin z /= /1Xk /=/0 /( /; /1/) k z /2 k /+/1 /(/2 k /+ /1/)/! /! sin A /= /1Xk /=/0 /( /; /1/) k A /2 k /+/1 /(/2 k /+ /1/)/! /;;ln /( I /+ z /)/= /1Xk /=/1 /( /; /1/) k /+/1 z k k /! ln /( I /+ A /)/= /1Xk /=/1 /( /; /1/) k /+/1 A k k /:It turns out that this approac h is not v ery practical for solving problems/.De/nition /2/./9/./1/. If A /2 C n / n/, f /( z /) is analytic in the op en domainD /;; /; is a closed simple line /(do es not cut itself /) in D and the sp ectrum / /( A /)of the matrix A is included in domain D/; enfolded b y /;/, thenf /( A /) def/= /1 /2 /i I/; f /( z /)/( zI /; A /) /; /1dz /;; /(/2/8/)where the in tegral is applied to the matrix elemen t b y elemen t/.Remark /2/./9/./1/. F orm ula /(/2/8/) is an analogue to the Cauc h y in tegralform ula pro v ed for functions of a complex v ariable/.Example /2/./9/./1/. Let f /( z /)/= z and A /= /"a b/0 c /#/: Chec kh o w to calculateb y rule /(/2/8/)/. Since //1 /= a and //2 /= c are the eigen v alues of A /, then let usc ho ose the line /; /: j z j /= r /;; where r/> max /( j a j /;; j c j /) /: The function f /( z /)/= z isanalytic in domain D/; /: First w e /nd/( zI /; A /) /; /1/= /"z /; a /; b/0 z /; c /#/; /1/=/= /"/1 /= /( z /; a /) /( b/= /( a /; c /)/)/(/1 /= /( z /; a /) /; /1 /= /( z /; c /)/)/0 /1 /= /( z /; c /) /#/;;/7/9 and thenf /( /"a b/0 c /#/)/= /1 /2 /i Ij z j /= r /"z/= /( z /; a /) /( b/= /( a /; c /)/)/( z/= /( z /; a /) /; z/= /( z /; c /)/)/0 /1 /= /( z /; c /) /#dz /=/= /"/1 /2 /i Hj z j /= r z/= /( z /; a /) dz /1 /2 /i Hj z j /= r /( b/= /( a /; c /)/)/( z/= /( z /; a /) /; z/= /( z /; c /)/) dz/0 /1 /2 /i Hj z j /= r /1 /= /( z /; c /) dz /= /#/= /"a b/0 c /#/:Prop osition /2/./9/./1/. If f /( A /)/= /( fk /;; j /) /;; thenfk /;; j /= /1 /2 /i I/; f /( z /) e Tk /( zI /; A /) /; /1ej dz /;;where ek is a v ector in space C nwhose k /; th comp onen t is one and theremaining ones are zeros/.Pr o of/. Let B /=/( bk /;; j /)/= /( zI /; A /) /; /1/: Then/,e Tk /( zI /; A /) /; /1ej /= e Tk B ej /= hbk /;;/1 /
/
/
bk /;; n iej /= bk /;; j /:Since the matrix is in tegrated elemen tb y elemen t/, w e obtain that/1 /2 /i I/; f /( z /) e Tk /( zI /; A /) /; /1ej dz /= /1 /2 /i I/; f /( z /) bk /;; j dz /= fk /;; j /: /2Prop osition /2/./9/./2/. If A /2 C n / n/, there /9 f /( A /) /;; i/.e/./, the conditions ofde/nition /2/./9/./1 are satis/ed/, andA /= XB X /; /1/= Xd i a g /( B/1 /;; /:/:/: /;;Bp /) X /; /1/;; Bk /2 C nk / nk/;; /(/2/9/)thenf /( A /)/= X f /( B /) X /; /1/= X diag /( f /( B/1 /) /;; /:/:/: /;; f /( Bp /)/) X /; /1/: /(/3/0/)Pr o of/. F rom /(/2/8/) /, /(/2/9/) and XX /; /1/= I w e /nd thatf /( A /)/= /1 /2 /i I/; f /( z /)/( zI /; A /) /; /1dz /= /1 /2 /i I/; f /( z /)/( Xz IX /; /1/; XB X /; /1/) /; /1dz /=/= /1 /2 /i I/; Xf /( z /)/( zI /; B /) /; /1X /; /1dz /= X /1 /2 /i I/; f /( z /)/( zI /; B /) /; /1dz X /; /1/= X f /( B /) X /; /1/:/8/0 Since/( Iz /; B /) /; /1/= diag /(/( Iz /; B/1 /) /; /1/;; /:/:/: /;; /( Iz /; Bp /) /; /1/)andf /( B /)/= /1 /2 /i I/; f /( z /)/( zI /; B /) /; /1dz /=/= diag /( /1 /2 /i I/; f /( z /)/( zI /; B/1 /) /; /1dz /;; /:/:/: /;; diag /( /1 /2 /i I/; f /( z /)/( zI /; Bp /) /; /1dz /)/=/= diag /( f /( B/1 /)/;; /:/:/: /;; f /( Bp /)/) /;;thenf /( A /)/= X f /( B /) X /; /1/= X diag /( f /( B/1 /) /;; /:/:/: /;; f /( Bp /)/) X /; /1/;;whic h w as to b e pro v ed/. /2Prop osition /2/./9/./3/. If A /2 C n / nand X /; /1AX /= diag /( J/1 /;; /:/:/: /;;Jp /)i st h eJordan normal form of the matrix A /, whereJi /= /2/6/6 /6 /6 /6/6/6 /6/4 /i /1 /0 /
/
/
/0/0 /i /1 /././. /./. /././. /. /././. /././. /././. /./. /./././. /././. /././. /./././1/0 /
/
/
/
/
/
/0 /i /3/7/7 /7 /7 /7/7/7 /7/5is an mi / mi Jordan blo c k/, m/1 /+ /:/:/: /+ mp /= n/;; and f /( z /) is analytic on anop en set that includes the sp ectrum / /( A /)o ft h e matrix A /, thenf /( A /)/= Xd i a g /( f /( J/1 /) /;; /:/:/: /;;f /( Jp /)/) X /; /1/;; /(/3/1/)wheref /( Ji /)/= /2/6/6 /6 /6 /6/6/6 /6/4 f /( /i /) f /0/( /i /) /
/
/
/
/
/
f /( mi /; /1/)/( /i /) /= /( mi /; /1/)/!/0 f /( /i /) /./././
/
/
/./. /./././. /././. /././. /././. /./././././. /././. /././. /././.f /0/( /i /)/0 /
/
/
/
/
/
/
/
/
f /( /i /) /3/7/7 /7 /7 /7/7/7 /7/5 /: /(/3/2/)Pr o o of/. Using Prop osition /2/./9/./2 it is su/cien t to consider only the v alueF /( G /)/, where G /= /I /+ E is a q / q Jordan blo c k and E /=/( /i /;; j /; /1 /) /: Let thematrix zI /; G b e regular/. SinceE k/=/( /i /;; j /; k /) /) /( k / q /) E k/=/0 /) /;;/8/1 then/( zI /; G /) /; /1/= q /; /1Xk /=/0 E k /( z /; / /) k /+/1andf /( G /)/= /1 /2 /i I/; f /( z /)/( zI /; G /) /; /1dz /= /1 /2 /i I/; f /( z /) q /; /1Xk /=/0 E k /( z /; / /) k /+/1 dz /=/= q /; /1Xk /=/0 E k /1 /2 /i I/; f /( z /) /( z /; / /) k /+/1 dz /= q /; /1Xk /=/0 f /( k /)/( / /) k /! E k/:No w/, taking in to accoun t the condition E k/=/( /i /;; j /; k /) /;; the assertion holds/. /2Example /2/./9/./2/. Find cos A ifA /= /2/6/6 /6/6/6/6/4 /0 /0 /1 /1 /1/0 /0 /0 /1 /1/0 /0 /0 /0 /1/0 /0 /0 /0 /0/0 /0 /0 /0 /0 /3/7/7 /7/7/7/7/5 /:Since //1 /= /:/:/: /= //5 /=/0 and the function cos z is analytic in the neigh/-b ourho o d of /0 /, then for the calculation of cos A w e can apply the algorithmgiv en in Prop osition /2/./9/./3/. W e use for the calculation of Jordan decomp osi/-tion of the matrix A /"Maple/"/:A /= XJ X /; /1/= /2/6/6 /6/6/6 /6/4 /1 /1 /1 /1 /0/0 /1 /0 /1 /0/0 /1 /0 /0 /0/0 /0 /0 /0 /1/0 /0 /1 /0 /0 /3/7/7 /7/7/7 /7/5 /2/6/6 /6/6/6 /6/4 /0 /1 /0 /0 /0/0 /0 /1 /0 /0/0 /0 /0 /0 /0/0 /0 /0 /0 /1/0 /0 /0 /0 /0 /3/7/7 /7/7/7 /7/5 /2/6/6 /6/6/6 /6/4 /1 /; /1 /0 /0 /; /1/0 /0 /1 /0 /0/0 /0 /0 /0 /1/0 /1 /; /1 /0 /0/0 /0 /0 /1 /0 /3/7/7 /7/7/7 /7/5 /:Therefore/, J /= diag /( J/1 /;;J/2 /) /;; whereJ/1 /= /2/6/4 /0 /1 /0/0 /0 /1/0 /0 /0 /3/7/5 /^ J/2 /= /"/0 /1/0 /0 /#/:Using /(/3/)/, w e /nd the matrices cos J/1 and cos J/2 /:cos J/1 /= /2/6/4 cos /0 /( /; sin /0/) /= /1/! /( /; cos /0/) /= /2/!/0 cos /0 /( /; sin /0/) /= /1/!/0 /0 cos /0 /3/7/5 /= /2/6/4 /1 /0 /; /1 /2/0 /1 /0/0 /0 /1 /3/7/5/8/2 andcos J/2 /= /"cos /0 /( /; sin /0/) /= /1/!/0 cos /0 /#/= /"/1 /0/0 /1 /#/:After that/, u s i n g/( /2 /) /,w e /nd the matrix w an ted/:cos A /= /2/6/6/6 /6 /6 /6/4 /1 /1 /1 /1 /0/0 /1 /0 /1 /0/0 /1 /0 /0 /0/0 /0 /0 /0 /1/0 /0 /1 /0 /0 /3/7/7/7 /7 /7 /7/5 /2/6/6/6 /6 /6 /6/4 /1 /0 /; /1 /2 /0 /0/0 /1 /0 /0 /0/0 /0 /1 /0 /0/0 /0 /0 /1 /0/0 /0 /0 /0 /1 /3/7/7/7 /7 /7 /7/5 /2/6/6/6 /6 /6 /6/4 /1 /; /1 /0 /0 /; /1/0 /0 /1 /0 /0/0 /0 /0 /0 /1/0 /1 /; /1 /0 /0/0 /0 /0 /1 /0 /3/7/7/7 /7 /7 /7/5 /=/= /2/6/6 /6 /6 /6 /6/4 /1 /0 /0 /0 /; /1 /2/0 /1 /0 /0 /0/0 /0 /1 /0 /0/0 /0 /0 /1 /0/0 /0 /0 /0 /1 /3/7/7 /7 /7 /7 /7/5 /:Corollary /2/./9/./1/. If A /2 C n / n/, A /= X diag /( //1 /;; /:/:/: /;;/n /) X /; /1and there/9 f /( A /) /;; thenf /( A /)/= X diag /( f /( //1 /) /;; /:/:/: /;;f /( /n /)/) X /; /1/:Pr o of/. This is a sp ecial case of Prop osition /2/./9/./3/. All the Jordan blo c ksare /1 / /1 /:Example /2/./9/./3/. If //1 /;; /:/:/: /;; /n are the eigen v alues of the matrix A /2C n / nand x/1 /;; /:/:/: /;; xn are the corresp onding linearly indep enden t eigen v ectors/, i/.e/./, x/1 /;; /:/:/: /;; xn generate a basis in C n/;; then X /= hx/1 /
/
/
xn i/;; and fromthe analyticit yo f e x p z/;; cos z/;; sin z in the whole /nite part of complex planeit follo ws thatexp A /= X diag /( exp //1 /;; /:/:/: /;; exp /n /) X /; /1/= X /(exp //) X /; /1/;;cos A /= X diag /(cos //1 /;; /:/:/: /;; cos /n /) X /; /1/= X /(cos / /) X /; /1/;;sin A /= Xd i a g /(sin //1 /;; /:/:/: /;; sin /n /) X /; /1/= X /(sin / /) X /; /1/;;where / /;; exp / /;; cos / /;; sin / /;; exp A/;; cos A/;; sin A /2 C n / nand//= /2/6/6/4 //1 /
/
/
/0/./. /. /././. /./. /./0 /
/
/
/n /3/7/7/5 /;; exp / /= /2/6/6/4 exp //1 /
/
/
/0/./. /. /././. /./. /./0 /
/
/
exp /n /3/7/7/5 /;;/8/3 cos / /= /2/6/6/4 cos //1 /
/
/
/0/././. /././. /./././0 /
/
/
cos /n /3/7/7/5 /;; sin / /= /2/6/6/4 sin //1 /
/
/
/0/././. /././. /./././0 /
/
/
sin /n /3/7/7/5 /:Next w e consider the problem arising in the appro ximation of the functionf /( A /) b y the function g /( A /) /: This kind of problem arises/, for example/, if w ereplace f /( A /) with its T a ylor p olynomial of degree q/:Prop osition /2/./9/./4/. Let A /2 C n / n/, X /; /1AX /= diag /( J/1 /;; /:/:/: /;;Jp /) /;; whereJi /= /2/6/6/6/6 /6 /6 /6/6/4 /i /1 /0 /
/
/
/0/0 /i /1 /././. /./././././. /././. /././. /././. /./././././. /././. /././. /./././1/0 /
/
/
/
/
/
/0 /i /3/7/7/7/7 /7 /7 /7/7/5is an mi / mi Jordan blo c k and m/1 /+ /:/:/: /+ mp /= n/: If the functions f /( z /) andg /( z /) are analytic on an op en set con taining the sp ectrum / /( A /) of the matrixA/;; thenk f /( A /) /; g /( A /) k/2 / k/2 /( X /) max/1 / i / p /^ /0 / r / mi /; /1 mi /// f /( r /)/( /i /) /; g /( r /)/( /i /) /// r /! /: /(/3/3/)Pr o of/. Cho osing h /( z /)/= f /( z /) /; g /( z /) w e ha v ejj f /( A /) /; g /( A /) jj/2 /= jj X diag /( h /( J/1 /) /;; /:/:/: /;;h /( Jp /)/) X /; /1jj/2 //j j X jj/2 jj diag /( h /( J/1 /) /;; /:/:/: /;;h /( Jp /)/) jj/2 jj X /; /1jj/2 / k/2 /( X /) max/1 / i / p jj h /( Ji /) jj/2 /:Using Prop osition /2/./9/./3 and inequalit y jj B jj/2 / n /
maxi/;; j j bi/;; j j w e /nd thatjj h /( Ji /) jj/2 / mi max/0 / r / mi /; /1 // / h /( r /)/( /i /) // / r /!and th us/, the assertion holds/. /2Example /2/./9/./4/. LetA /= /2/6/4 /1 /= /1/0 /1 /= /1/0 /0/0 /1 /= /1/0 /1 /= /1/0/0 /0 /1 /= /1/0 /3/7/5 /:/8/4 W e estimate the di/erence sin A /; A/:Since //1 /= //2 /= //3 /= /: /1 and the functions f /( z /) /= sin z and g /( z /)/= z areanalytic in the neigh b ourho o d of /: /1 /;; then w e can apply the estimation /(/3/3/)obtained in Prop osition /2/./9/./4/. First/, w e use /"Maple/" for /nding the Jordandecomp osition of the matrix A /:A /= XJ X /; /1/= /2/6/4 /1 /1/0/0 /0 /1/0 /1 /1/0 /0/0 /0 /1 /3/7/5 /2/6/4 /1 /1/0 /1 /0/0 /1 /1/0 /1/0 /0 /1 /1/0 /3/7/5 /2/6/4 /1/0/0 /0 /; /1/0/0/0 /1/0 /0/0 /0 /1 /3/7/5 /:Hence/, there is only one Jordan blo c k in the Jordan decomp osition of thematrix A /, i/.e/.J /= J/1 /= diag /( J/1 /) /:Second/, w e use /"Maple/" for /nding the condition n um be r of X /:k/2 /( X /) / /2/0/0 /: /0/1 /:Sincef /( z /) /; g /( z /)/= s i n z /; z /)j f /( /: /1/) /; g /( /: /1/) j /= /0/! /= j sin /: /1 /; /: /1 j/ /1 /: /6/6/5/8 / /1/0 /; /4/;;f /0/( z /) /; g /0/( z /)/=c o s z /; /1 /)j f /0/(/1/) /; g /0/(/1/) j /= /1/! /= j cos /: /1 /; /1 j/ /4 /: /9/9/5/8 / /1/0 /; /3andf /0/0/( z /) /; g /0/0/( z /)/= /; sin z /)j f /0/0/(/1/) /; g /0/0/(/1/) j /= /2/! /= j/; sin /: /1 j /= /2 / /4 /: /9/9/1/7 / /1/0 /; /2/;;then/, b y estimation /(/3/0/)/, w e ha v e thatjj sin A /; A jj/2 / /2/0/0 /: /0/1 /
/3 /
/4 /: /9/9/1/7 / /1/0 /; /2/ /2/9 /: /9/5/2 /:It is kno wn that matrix X in the Jordan decomp osition of A is not uniquelydetermined/. W e try to c ho ose the matrix X so that the condition n um be rk/2 /( X /) should b e minimal/. Applying the Filip o v algorithm to /nd the Jordandecomp osition of A /(see Prop osition /2/./5/./2/./1/)/, w e obtain thatA /= XJ X /; /1/= /2/6/4 /1 /0 /0/0 /1/0 /0/0 /0 /1/0/0 /3/7/5 /2/6/4 /1 /1/0 /1 /0/0 /1 /1/0 /1/0 /0 /1 /1/0 /3/7/5 /2/6/4 /1 /0 /0/0 /1 /1/0 /0/0 /0 /1 /1/0/0 /3/7/5 /;;/8/5 wherek/2 /( X /) /= /1/0/0 /:It turns out thatA /= XJ X /; /1/= /2/6/4 /1 /1/0 /0 /0/0 /1 /0/0 /0 /1/0 /3/7/5 /2/6/4 /1 /1/0 /1 /0/0 /1 /1/0 /1/0 /0 /1 /1/0 /3/7/5 /2/6/4 /1/0 /0 /0/0 /1 /0/0 /0 /1 /1/0 /3/7/5is also a Jordan decomp osition of A /, wherek/2 /( X /)/=/1 /0 /:Therefore/, the b est estimation w e can ha v e b y Prop osition /2/./9/./4 isjj sin A /; A jj/2 / /1/0 /
/3 /
/4 /: /9/9/1/7 / /1/0 /; /2/ /1 /: /4/9/7/5 /:Otherwise/, in this example for calculating sin A w e can apply the algorithmgiv en in Prop osition /2/./9/./3/. Using the form ula /(/3/2/)/, w e ha v e thatsin J /= /2/6/4 sin /: /1 /(cos /: /1/) /= /1/! /( /; sin /: /1/) /= /2/!/0 sin /: /1 /(cos /: /1/) /= /1/!/0 /0 sin /: /1 /3/7/5 /=/= /2/6/4 /: /0/9/9/8/3/3 /: /9/9/5 /; /: /0/4/9/9/1/7/0 /: /0/9/9/8/3/3 /: /9/9/5/0 /0 /: /0/9/9/8/3/3 /3/7/5 /:By form ula /(/3/1/) w e calculate the v alue of the function in question/:sin A /= /2/6/4 /1 /1/0/0 /0 /1/0 /1 /1/0 /0/0 /0 /1 /3/7/5 /2/6/4 /: /0/9/9/8/3/3 /: /9/9/5 /; /: /0/4/9/9/1/7/0 /: /0/9/9/8/3/3 /: /9/9/5/0 /0 /: /0/9/9/8/3/3 /3/7/5 /2/6/4 /1/0/0 /0 /; /1/0/0/0 /1/0 /0/0 /0 /1 /3/7/5 /=/= /2/6/4 /: /0/9/9/8/3/3 /: /0/9/9/5 /; /: /0/0/0/4/9/9/1/7/0 /: /0/9/9/8/3/3 /: /0/9/9/5/0 /0 /: /0/9/9/8/3/3 /3/7/5 /:Hence/,sin A /; A /= /2/6/4 /: /0/9/9/8/3/3 /; /: /1 /: /0/9/9/5 /; /: /1 /; /: /0/0/0/4/9/9/1/7/0 /: /0/9/9/8/3/3 /; /: /1 /: /0/9/9/5 /; /: /1/0 /0 /: /0/9/9/8/3/3 /; /: /1 /3/7/5 /=/8/6 /= /2/6/4 /; /1 /: /6/7 / /1/0 /; /4/; /: /0/0/0/5 /; /4 /: /9/9/1/7 / /1/0 /; /4/0 /; /1 /: /6/7 / /1/0 /; /4/; /: /0/0/0/5/0 /0 /; /1 /: /6/7 / /1/0 /; /4 /3/7/5andk sin A /; A k/2 / /8 /: /8/0/9/8 / /1/0 /; /4/:As a result of this example/, w e can assert that estimation /(/3/3/) pro v ed inProp osition /2/./9/./5 is quite rough/.Prop osition /2/./9/./5/. If the Maclaurin expansion of the function f /( z /)f /( z /)/= /1Xk /=/0 ck z kis con v ergen t in the circle con taining the sp ectrum / /( A /) of the matrix A /2C n / n/;; thenf /( A /)/= /1Xk /=/0 ck A k/:Pr ove this assertion with an additioal assumption that the matrix A hasa basis consisting of its eigen v ectors/. In this case/, b y Corollary /2/./9/./1/,f /( A /)/= Xd i a g /( f /( //1 /) /;; /:/:/: /;;f /( /n /)/) X /; /1/=/= Xd i a g /( /1Xk /=/0 ck / k/1 /;; /:/:/: /;; /1Xk /=/0 ck / kn /) X /; /1/=/= X / /1Xk /=/0 ck D k /!X /; /1/= /1Xk /=/0 ck /( XD X /; /1/) k/= /1Xk /=/0 ck A k/: /2Prop osition /2/./9/./6/. If the Maclaurin series of the function f /( z /)f /( z /)/= /1Xk /=/0 ck z kis con v ergen t in the circle con taining the sp ectrum / /( A /) of the matrix A /2C n / n/;; thenjj f /( A /) /; qXk /=/0 ck A kjj/2 / n /( q /+ /1/)/! max/0 / s / /1 jj A q /+/1f /( q /+/1/)/( As /) jj/2 /:/8/7 Pr o of/. Let us de/ne the matrix E /( s /)b yf /( As /)/= qXk /=/0 ck /( As /) k/+ E /( s /) /(/0 / s / /1/) /: /(/3/4/)If fi /;; j /( s /)/= /[ f /( As /)/]i /;; j /,t h e n fi /;; j /( s /) is analytic/, and therefore/,fi/;; j /( s /)/= qXk /=/0 fi /;; j /(/0/) k /! s k/+ f /( q /+/1/)i /;; j /( /"i /;; j /) /( q /+ /1/)/! s q /+/1/;; /(/3/5/)where /0 / /"i /;; j / s / /1 /: By comparing the po w ers of the v ariable s in /(/3/4/)and /(/3/5/)/, w e conclude that /[ E /( s /)/]i /;; j has the form/"i /;; j /( s /)/= f /( q /+/1/)i /;; j /( /"i /;; j /) /( q /+ /1/)/! s q /+/1/:If f /( q /+/1/)i /;; j /=/[ A q /+/1f /( q /+/1/)/( As /)/]i /;; j /;; thenj /"i /;; j /( s /) j/ max/0 / s / /1 j f /( q /+/1/)i /;; j /( /"i /;; j /) j /( q /+ /1/)/! / n max/0 / s / /1 jj A q /+/1f /( q /+/1/)/( As /) jj/2 /: /2Exercise /2/./9/./1/. Pro v e that for an arbitrary matrix A /2 C n / nI /+ cos/(/2 A /) /= /2 cos /2Aandsin/(/2 A /) /= /2 sin A /
cos A/:Exercise /2/./9/./2/. Apply Prop osition /2/./9/./6 to the estimation of the errorsin the appro ximate equalitiessin A / qXk /=/0 /( /; /1/) k A /2 k /+/1 /(/2 k /+ /1/)/!andcos A / qXk /=/0 /( /; /1/) k A /2 k /(/2 k /)/! /:Prop osition /2/./9/./7 /( Sylv ester theorem /)/. If all eigen v alues /k of thematrix A /2 C n / nare di/eren t/, thenf /( A /)/= nXk /=/1 f /( /k /) /i /6/= k /( A /; /i I /) /i /6/= k /( /k /; /i /) /(/3/6/)/8/8 orf /( A /)/= /1 /
nXk /=/1 /
k A k /; /1/;; /(/3/7/)where /
k /( k /= /1/;; /2/;; /:/:/: /;; n /) is the determinan t obtained from the V ander/-monde determinan t/
/= // / /// / / / /1 /1 /:/:/: /1//1 //2 /:/:/: /n/:/:/: /:/:/: /:/:/: /:/:/:/ n /; /1/1 / n /; /1/2 /:/:/: / n /; /1n // / /// / / /b y replacing the k /-th ro w v ector/( / k /; /1/1 / k /; /1/2 /:/:/: / k /; /1/1 /)b y the v ector/( f /( //1 /) f /( //2 /) /:/:/: f /( /n /)/) /:Example /2/./9/./4/. Calculate exp A if A /= /"/1 /1/; /1 /1 /#/:First w e /nd the eigen v alues of A /:// / / / /1 /; / /1/; /1 /1 /; / // / / / /=/0 /) /(/1 /; / /) /2/+/1 /= /0 /) /(//1 /=/1 /+ i//2 /=/1 /; iThen w e use form ula /(/3/6/)/:exp /"/1 /1/; /1 /1 /#/= A /; /(/1 /; i /) I /1/+ i /; /1/+ i exp /(/1 /+ i /)/+ A /; /(/1 /+ i /) I /1 /; i /; /1 /; i exp /(/1 /; i /)/=/= /"i /1/; /1 i /# /2 i exp /(/1 /+ i /)/+ /"/; i /1/; /1 /; i /# /; /2 i /=/= e /
/"/(exp i /+ exp /( /; i /)/) /= /2 /(exp i /; exp /( /; i /)/) /= /2 i/; /(exp i /; exp /( /; i /)/) /= /2 i /(exp i /+ exp /( /; i /)/) /= /2 /#/= e /
/"cos /1 sin /1/; sin /1 cos /1 /#No w applying /(/3/7/)/, w e ha v eexp /"/1 /1/; /1 /1 /#/= /(/1 /= det /"/1 /1/1/+ i /1 /; i /#/)/[ I det /"exp /(/1 /+ i /) exp /(/1 /; i /)/1/+ i /1 /; i /#/+/+ /"/1 /1/; /1 /1 /#det /"/1 /1exp /(/1 /+ i /) exp /(/1 /; i /) /#/]/8/9 /= e /; /2 i f /"/1 /0/0 /1 /#/[/(/1 /; i /)e x p i /; /(/1 /+ i /) exp /( /; i /)/] /+ /"/1 /1/; /1 /1 /#/[exp /( /; i /) /; exp i /] g /=/= e /; /2 i /"/; i exp i /; i exp /( /; i /) exp /( /; i /) /; exp i/; exp /( /; i /)/+ e x p i /; i exp i /; i exp /( /; i /) /#/= e /
/"cos /1 sin /1/; sin /1 cos /1 /#/:W e solv e this problem once more using the form ula exp A /= S exp / S /; /1/;;where S is the matrix formed of the eigen v alues of A/: Find the eigen v aluesof A /://1 /=/1 /+ i /) /2/4 /; i /1 /././. /0/; /1 /; i /././. /0 /3/5/) x/1 /= /"/1i /#and//2 /=/1 /; i /) /2/4 i /1 /./. /. /0/; /1 i /./. /. /0 /3/5/) x/2 /= /1/; i /;;and the matrixS /= /"/1 /1i /; i /#/:Henceexp A /= S exp / S /; /1/= /"/1 /1i /; i /#/"e /1/+ i/0/0 e /1 /; i /#/"/1 /= /2 /; i/= /2/1 /= /2 i/= /2 /#/=/= /"e /1/+ ie /1 /; iie /1/+ i/; ie /1 /; i /#/"/1 /= /2 /; i/= /2/1 /= /2 i/= /2 /#/=/= e /"/(exp i /+ exp /( /; i /)/) /= /2 /(exp i /; exp /( /; i /)/) /= /2 i/; /(exp i /; exp /( /; i /)/) /= /2 i /(exp i /+ exp /( /; i /)/) /= /2 /#/=/= e /"cos i sin i/; sin i cos i /#/:/2 Computation Metho ds in Linear Algebra/2/./1 LU/-F actorization/2/./1/./1 Solution of T riangular Systems/9/0 Let us consider the solution of a /2 / /2 /; lo w er triangular system/"l/1/1 /0l/2/1 l/2/2 /#/"//1//2 /#/= /"b/1b/2 /#/( l/1/1 l/2/2 /6/=/0 /)b y forw ard substitution/. F rom the /rst equation w e obtain //1 /= b/1 /=l/1/1 /;; andthen from the second one //2 /=/( b/2 /; l/2/1 //1 /) /=l/2/2 /:Prop osition /1/./1/./1 /( forwar d substitution /)/. If L /= /( lij /) /2 R n / nis alo w er triangular matrix/, Qni /=/1 lii /6/=/0 and L x /= b /;; then the solution is/i /=/( bi /; i /; /1Xk /=/1 lik /k /) /=lii /( i /=/1 /: n /) /:Solv e the /2 / /2 /; upp er triangular system/"u/1/1 u/1/2/0 u/2/2 /#/"//1//2 /#/= /"b/1b/2 /#/( u/1/1 u/2/2 /6/=/0 /)b y bac k substitution/. F rom the second equation w e obtain //2 /= b/2 /=u/2/2 /;; andthen from the /rst one //1 /=/( b/1 /; u/1/2 //1 /) /=u/1/1 /:Prop osition /1/./1/./2 /( b ack substitution /)/. If U /=/( uij /) /2 R n / nis an upp ertriangular matrix/, Qni /=/1 uii /6/=/0 and U x /= b /;; then the solution is/i /=/( bi /; nXk /= i /+/1 uik /k /) /=uii /( i /=/1 /: n /) /:In the case of forw ard substitution as w ell as in the case of bac k sub/-stitution the solution of the system with a regular n / n /; triangular matrixrequires /1/+/3/+ /:/:/: /+/( /2 n /; /1/) /= n /2op erations/.Prop osition /1/./1/./3 /( forw ard substitution/: ro w v ersion/)/. If L /2 R n / nislo w er triangular/, Qni /=/1 lii /6/= /0 /, L x /= b /;; and //1 has b een found/, then aftersubstitution of //1 in to the equations from the second to the n /-th/, w e obtaina new /( n /; /1/) / /( n /; /1/) /; lo w er triangular systemL /(/2 /: n /;; /2/: n /) x /(/2 /: n /)/= b /(/2 /: n /) /; x /(/1/) L /(/2 /: n /;;/1 /) /:Prop osition /1/./1/./4 /( bac k substitution/:column v ersion/)/. If U /2 R n / nisupp er triangular/, Qni /=/1 uii /6/= /0 /, U x /= b /;; and /n has b een found/, then after/9/1 the substitution of /n in to the equations from the /rst to the /( n /; /1/)/-th/, w eobtain a new /( n /; /1/) / /( n /; /1/) /; upp er triangular systemU /( /1/:/( n /; /1/)/;; /1/: /( n /; /1/)/) x /(/1 /: /( n /; /1/)/) /= b /( /1/:/( n /; /1/)/) /; x /( n /) U /( /1/:/( n /; /1/)/;; n /) /:No w w e consider the sim ultaneous solution of sev eral systems with acommon system matrix/. Let us consider the system LX /= B/;; where L /2R n / nis a regular lo w er triangular matrix/, B /2 R n / qand the unkno wn isX /2 R n / q/. W e represen t this system in blo c k form/2/6/6 /6 /6/4 L/1/1 /0 /
/
/
/0L/2/1 L/2/2 /
/
/
/0/./. /. /./. /. /././. /./. /.LN /1 LN /2 /
/
/
LNN /3/7/7 /7 /7/5 /2/6/6 /6 /6/4 X/1X/2/./. /.XN /3/7/7 /7 /7/5 /= /2/6/6 /6 /6/4 B/1B/2/./. /.BN /3/7/7 /7 /7/5 /;; /(/1/)where the diagonal blo c ks are square/. F rom the equation L/1/1 X/1 /= B/1 w ecan /nd X/1 /. By using for system /(/1/) the ro w v ersion giv en in Prop osition/1/./1/./3/, w e obtain/2/6/6/6 /6/4 L/2/2 /0 /
/
/
/0L/3/2 L/3/3 /
/
/
/0/./. /. /./. /. /././. /./. /.LN /2 LN /3 /
/
/
LNN /3/7/7/7 /7/5 /2/6/6/6 /6/4 X/2X/3/./. /.XN /3/7/7/7 /7/5 /= /2/6/6/6 /6/4 B/2 /; L/2/1 X/1B/3 /; L/3/1 X/1/./. /.BN /; LN /1 X/1 /3/7/7/7 /7/5 /:Con tin uing in this w a y w e obtain the solution of system /(/1/)/.Prop osition /1/./1/./5/. T riangular matrices ha v e the follo wing prop erties/:/ the in v erse of an upp er /(lo w er/) triangular matrix is upp er /(lo w er/) tri/-angular/;;/ the pro duct of t w o upp er /(lo w er/) triangular matrices is upp er /(lo w er/)triangular/;;/ the in v erse of an unit upp er /(lo w er/) triangular matrix is unit upp er/(lo w er/) triangular/;;/ the pro duct of t w o unit upp er /(lo w er/) triangular matrices is upp er/(lo w er/) triangular/./ Exercise /1/./1/./1/./* Pro v e Prop osition /1/./1/./5/./9/2 /2/./1/./2 Gauss T ransformation and LU/-F actorizationUnder certain conditions the system matrix A of the equation A x /= bcan be expressed in the form of a pro duct of a unit lower triangular matrixL with units on the main diagonal and an upp er triangular matrix U /,and asthe result/, one has to solv e t w o systems with triangular matrices/.Prop osition /1/./2/./1 /( LU /; metho d /)/. If A /2 R n / n/;; A /= LU/;; where Lis unit lo w er triangular/, U is regular upp er triangular and A x /= b /;; thenLU x /= b /;; and for the solution of the system one has /rst to solv e the systemL y /= b and then the system U x /= y /:Example /1/./2/./1/. Solv e the system/"/1 /2/3 /4 /#x /= /"/1/5 /#using LU /-metho d/. Since/"/1 /2/3 /4 /#/= /"/1 /0/3 /1 /#/"/1 /2/0 /; /2 /#/;;then/, b y Prop osition /1/./2/./1/, w e ha v e to solv e /rst the system/"/1 /0/3 /1 /#/"//1//2 /#/= /"/1/5 /#/:The solution of this system is //1 /= /1 and //2 /=/5 /; /3 /
/1/= /2 /: Second/, solvingthe system/"/1 /2/0 /; /2 /#/"//1//2 /#/= /"/1/2 /#/;;w e /nd that //2 /= /; /1 and //1 /=/1 /; /2 /
/( /; /1/) /= /3 /: Th us/, x /= h/3 /; /1 iT/:The Gaussian elimination metho d considered in the main course of linearalgebra for solution of systems of linear equations is also applicable also tothe LU /-factorization/. Let x /2 R m/;; where /k /6/=/0 /: If/i /= /i /=/k /( i /=/( k /+/1 /) /: m /) t /( k /)/= h/0 /
/
/
/0 /k /+/1 /
/
/
/m ik zeros/9/3 andMk /= I /; t /( k /)e Tk /;; /(/2/)thenMk x /= /2/6/6/6 /6 /6 /6 /6/6/6 /6/4 /1 /
/
/
/0 /0 /
/
/
/0/./. /. /././. /./. /. /./. /. /./. /./0 /1 /0 /0/0 /; /k /+/1 /1 /0/././. /././. /././. /./././0 /
/
/
/; /m /0 /
/
/
/1 /3/7/7/7 /7 /7 /7 /7/7/7 /7/5 /2/6/6/6 /6 /6 /6 /6/6/6 /6/4 //1/./. /./k/k /+/1/./././m /3/7/7/7 /7 /7 /7 /7/7/7 /7/5 /= /2/6/6/6 /6 /6 /6 /6/6/6 /6/4 //1/./. /./k/0/./././0 /3/7/7/7 /7 /7 /7 /7/7/7 /7/5 /:De/nition /1/./2/./1/. A matrix Mk of the form /(/2/) is called a Gauss matrix /,the comp onen ts t /(/( k /+/1 /) /: n /) are called Gauss multipliers/, and the v ectort /( k /)is called the Gauss ve ctor/. The transformation de/ned with the Gaussmatrix Mk is called the Gauss tr ansformation /.De/nition /1/./2/./2/. The v aluedk /= /(a/1/1 /;; if k /=/1 /;;det/( A /(/1 /: k/;; /1/: k /)/) /= det/( A /(/1 /: k /; /1 /;; /1/: k /; /1/)/) /;; if k /=/2 /: p/;;is called the k /; th pivot of the matrix A /2 R m / n/;; where p /= min /( m/;; n /) anddet/( A /(/1 /: i/;; /1/: i /)/) /6/=/0 /( i /=/1 /: p /; /1/) /:If A /2 R n / n/;; then for the nonzero piv ots of A the Gauss matrices M/1 /;; /:/:/: /;;Mn /; /1can b e found suc h that Mn /; /1 Mn /; /2 /
/
/
M/2 M/1 A /= U is upp er triangular/.Example /1/./2/./2/. Let us consider the /nding of the Gauss matrices M/1and M/2 and the upp er triangular matrix U forA /= /2/6/4 /2 /2 /; /1/4 /5 /2/; /2 /1 /2 /3/7/5By relation /(/2/)/, w e obtain thatM/1 /= I /; t /(/1/)e T/1 /= /2/6/4 /1 /0 /0/0 /1 /0/0 /0 /1 /3/7/5 /; /2/6/4 /0/4 /= /2/( /; /2/) /= /2 /3/7/5 h/1 /0 /0 i/=/= /2/6/4 /1 /0 /0/0 /1 /0/0 /0 /1 /3/7/5 /; /2/6/4 /0 /0 /0/2 /0 /0/; /1 /0 /0 /3/7/5 /= /2/6/4 /1 /0 /0/; /2 /1 /0/1 /0 /1 /3/7/5 /:/9/4 Th us/,M/1 A /= /2/6/4 /1 /0 /0/; /2 /1 /0/1 /0 /1 /3/7/5 /2/6/4 /2 /2 /; /1/4 /5 /2/; /2 /1 /2 /3/7/5 /= /2/6/4 /2 /2 /; /1/0 /1 /4/0 /3 /1 /3/7/5andM/2 /= I /; t /(/2/)e T/2 /= /2/6/4 /1 /0 /0/0 /1 /0/0 /0 /1 /3/7/5 /; /2/6/4 /0/0/3 /3/7/5 h/0 /1 /0 i/=/= /2/6/4 /1 /0 /0/0 /1 /0/0 /0 /1 /3/7/5 /; /2/6/4 /0 /0 /0/0 /0 /0/0 /3 /0 /3/7/5 /= /2/6/4 /1 /0 /0/0 /1 /0/0 /; /3 /1 /3/7/5 /:Therefore/,U /= M/2 M/1 A /= /2/6/4 /1 /0 /0/0 /1 /0/0 /; /3 /1 /3/7/5 /2/6/4 /2 /2 /; /1/0 /1 /4/0 /3 /1 /3/7/5 /= /2/6/4 /2 /2 /; /1/0 /1 /4/0 /0 /; /1/1 /3/7/5 /:Note that matrix A /( k /; /1/)/= Mk /; /1 /
/
/
M/1 A is upp er triangular in columns/1t o k /-/1/, and for the calculation of the elemen ts of the Gauss matrix Mk w euse the matrix v ector A /( k /; /1/)/( k /: m/;; k /) /: The calculation of Mk is p ossible ifa /( k /; /1/)kk /6/=/0 /. Moreo v er/, M /; /1k /= I /+ t /( k /)e Tk /: If to c ho oseL /= M /; /1/1 /
/
/
M /; /1n /; /1 /;;thenA /= LU/:W e stress that in our treatmen t the lo w er triangular matrix L is a unit lo w ertriangular matrix/.Prop osition /1/./2/./2/. If det /( A /(/1/: k /,/1 /: k /)/) /6/= /0 for /( k /=/1/: n /-/1/) /;; then A /2R n / nhas an LU factorization/. If the LU factorization exists and A is regular/,then the LU factorization is unique and det/( A /)/= u/1/1 /
/
/
unn /:Pr o of/. Supp ose k /; /1 steps ha v e b een tak en and the matrix A /( k /; /1/)/=Mk /; /1 /
/
/
M/1 A has b een found/. The elemen t a /( k /; /1/)kk is the k /-th piv ot of A anddet/( A /(/1/: k /,/1 /: k /)/) /= a /( k /; /1/)/1/1 /
/
/
a /( k /; /1/)kk /: Hence/, if A /(/1/: k /,/1 /: k /) is regular/, thena /( k /; /1/)kk /6/=/0 /;; and A has an LU factorization/. Let us supp ose that the regularmatrix A has t w o LU factorizations A /= L/1 U/1 and A /= L/2 U/2 /: W e ha v e/9/5 L/1 U/1 /= L/2 U/2 or L /; /1/2 L/1 /= U/2 U /; /1/1 /: Since L /; /1/2 L/1 is unit lo w er triangular andU/2 U /; /1/1 is upp er triangular/, then L /; /1/2 L/1 /= I /, U/2 U /; /1/1 /= I and L/2 /= L/1 andalso U/2 /= U/1 /: /2Example /1/./2/./3/. /Find the LU factorization of the matrixA /= /"/2 /1/8 /7 /#/:Find the Gauss matrix M/1 for A /:M/1 /= I /; t /(/1/)e T/1 /= /"/1 /0/0 /1 /#/; /"/0/8 /= /2 /#h/1 /0 i/=/= /"/1 /0/0 /1 /#/; /"/0 /0/4 /0 /#/= /"/1 /0/; /4 /1 /#/:Th us/,M/1 A /= /"/1 /0/; /4 /1 /#/"/2 /1/8 /7 /#/= /"/2 /1/0 /3 /#/;; M /; /1/1 /= /"/1 /0/4 /1 /#andL /= /"/1 /0/4 /1 /#/;; U /= /"/2 /1/0 /3 /#/;; A /= LU /= /"/1 /0/4 /1 /#/"/2 /1/0 /3 /#/:Example /1/./2/./4/. /Find the LU factorization ofA /= /2/6/4 /2 /3 /3/0 /5 /7/6 /9 /8 /3/7/5 /:Find the Gauss matrix M/1 for A /:M/1 /= I /; t /(/1/)e T/1 /= /2/6/4 /1 /0 /0/0 /1 /0/0 /0 /1 /3/7/5 /; /2/6/4 /0/0/6 /= /2 /3/7/5 h/1 /0 /0 i/=/= /2/6/4 /1 /0 /0/0 /1 /0/0 /0 /1 /3/7/5 /; /2/6/4 /0 /0 /0/0 /0 /0/3 /0 /0 /3/7/5 /= /2/6/4 /1 /0 /0/0 /1 /0/; /3 /0 /1 /3/7/5 /:/9/6 Th us/,M/1 A /= /2/6/4 /1 /0 /0/0 /1 /0/; /3 /0 /1 /3/7/5 /2/6/4 /2 /3 /3/0 /5 /7/6 /9 /8 /3/7/5 /= /2/6/4 /2 /3 /3/0 /5 /7/0 /0 /; /1 /3/7/5 /;; M /; /1/1 /= /2/6/4 /1 /0 /0/0 /1 /0/3 /0 /1 /3/7/5and since M/1 A is upp er triangular/, then M/2 /= I andL /= /2/6/4 /1 /0 /0/0 /1 /0/3 /0 /1 /3/7/5 /;; U /= /2/6/4 /2 /3 /3/0 /5 /7/0 /0 /; /1 /3/7/5 /;;A /= LU /= /2/6/4 /1 /0 /0/0 /1 /0/3 /0 /1 /3/7/5 /2/6/4 /2 /3 /3/0 /5 /7/0 /0 /; /1 /3/7/5 /:Example /1/./2/./5/. /By using the LU factorization/, solv e the system A x /= b /,whereA /= /2/6/4 /2 /3 /3/0 /5 /7/6 /9 /8 /3/7/5 /^ b /= /2/6/4 /2/2/5 /3/7/5 /:In example /1/./2/./4 w e found the LU factorization for A /:A /= LU /= /2/6/4 /1 /0 /0/0 /1 /0/3 /0 /1 /3/7/5 /2/6/4 /2 /3 /3/0 /5 /7/0 /0 /; /1 /3/7/5 /:By solving the system L y /= b /, i/.e/./,/2/6/4 /1 /0 /0/0 /1 /0/3 /0 /1 /3/7/5 /2/6/4 //1//2//3 /3/7/5 /= /2/6/4 /2/2/5 /3/7/5 /;;w e obtainy /= /2/6/4 //1//2//3 /3/7/5 /= /2/6/4 /2/2/; /1 /3/7/5 /:By solving the system U x /= y /, i/.e/./,/2/6/4 /2 /3 /3/0 /5 /7/0 /0 /; /1 /3/7/5 /2/6/4 //1//2//3 /3/7/5 /= /2/6/4 /2/2/; /1 /3/7/5 /;;/9/7 w e obtain thatx /= /2/6/4 //1//2//3 /3/7/5 /= /2/6/4 /1/; /1/1 /3/7/5 /:Exercise /1/./2/./1/. /Find the LU factorization of A ifa /) A /= /"/3 /1/6 /7 /#/;; b /) A /= /"/1 /0/8 /1 /#/;; c /) A /= /2/6/4 /1 /; /1 /0/; /1 /2 /; /1/0 /; /1 /1 /3/7/5 /:Exercise /1/./2/./2/. /By using the LU factorization solv e the system A x /= b /,whereA /= /2/6/4 /1 /; /1 /0/; /1 /2 /; /1/0 /; /1 /1 /3/7/5 /^ b /= /2/6/4 /2/; /3/4 /3/7/5 /:If the principal minors of a rectangular matrix A /2 R m / nare nonzero/,i/.e/./,det/( A /(/1 /: k/;; /1/: k /) /6/=/0 /( k /=/1 /:m i n /( m/;; n /)/) /;;then A has an LU factorization/.Example /1/./2/./6/. The follo wing equalities hold/:/2/6/4 /2 /1/8 /6/4 /5 /3/7/5 /= /2/6/4 /1 /0/4 /1/2 /3 /= /2 /3/7/5 /"/2 /1/0 /2 /#/;;/"/2 /8 /4/1 /6 /5 /#/= /"/1 /0/1 /= /2 /1 /#/"/2 /8 /4/0 /2 /3 /#/:As is kno wn from the main course of algebra/, the direct application of theGaussian elimination/, therefore also the direct realization of the LU factor/-ization fails/, if at least one of the principal minors is singular/. It turns outthat for a regular matrix it is p ossible after an appropriate in terc hange ofmatrix ro ws to /nd the LU factorization/. P erm utation matrices are used forin terc hanging the matrix ro ws /(columns/)/.De/nition /1/./2/./3/. A p ermutation matrix P /2 R n / nis the iden tit y Iwith its ro ws reordered/./9/8 Example /1/./2/./7/. Consider the e/ect of m ultiplying a /4 / /4 matrix A b ya concrete pe r m utation matrix P /.PA /= /2/6/6/6/4 /0 /0 /1 /0/0 /0 /0 /1/0 /1 /0 /0/1 /0 /0 /0 /3/7/7/7/5 /2/6/6/6/4 a/1/1 a/1/2 a/1/3 a/1/4a/2/1 a/2/2 a/2/3 a/2/4a/3/1 a/3/2 a/3/3 a/3/4a/4/1 a/4/2 a/4/3 a/4/4 /3/7/7/7/5 /= /2/6/6/6/4 a/3/1 a/3/2 a/3/3 a/3/4a/4/1 a/4/2 a/4/3 a/4/4a/2/1 a/2/2 a/2/3 a/2/4a/1/1 a/1/2 a/1/3 a/1/4 /3/7/7/7/5Multiplying b y the p erm utation matrix P on the left/, w e obtain a new matrix/,where the ro ws of initial the matrix are reordered exactly in the same w a yas the ro ws of the iden tit y I are reordered for getteing P/: Multiplying on therigh t/,AP /= /2/6/6/6/4 a/1/1 a/1/2 a/1/3 a/1/4a/2/1 a/2/2 a/2/3 a/2/4a/3/1 a/3/2 a/3/3 a/3/4a/4/1 a/4/2 a/4/3 a/4/4 /3/7/7/7/5 /2/6/6/6/4 /0 /0 /1 /0/0 /0 /0 /1/0 /1 /0 /0/1 /0 /0 /0 /3/7/7/7/5 /= /2/6/6/6/4 a/1/4 a/1/3 a/1/1 a/1/2a/2/4 a/2/3 a/2/1 a/2/2a/3/4 a/3/3 a/3/1 a/3/2a/4/4 a/4/3 a/4/1 a/4/2 /3/7/7/7/5 /;;w e obtain a new matrix/, where the columns of the initial matrix are reorderedin the same w a y as the columns of the iden tit y I are reordered for getting P /.The follo wing holdsProp osition /1/./2/./3/. If A /2 R n / nand det/( A /) /6/= /0 /;; then there exists ap erm utation matrix P /2 R n / nsuc h that all the principal minors of PA arenonzero/, and consequen tly /, there exists the LU factorizationPA /= LU/:Example /1/./2/./8/./* LetA /= /2/6/4 /0 /; /2 /2/1 /2 /; /1/3 /5 /; /8 /3/7/5 /:Find for a certain p erm utation matrix P /2 R /3 / /3the LU factorization of PA /.In terc hange the /rst and second ro ws of A /, i/.e/./, c ho oseP /= /2/6/4 /0 /1 /0/1 /0 /0/0 /0 /1 /3/7/5/9/9 and /nd for the matrixPA /= /2/6/4 /0 /1 /0/1 /0 /0/0 /0 /1 /3/7/5 /2/6/4 /0 /; /2 /2/1 /2 /; /1/3 /5 /; /8 /3/7/5 /= /2/6/4 /1 /2 /; /1/0 /; /2 /2/3 /5 /; /8 /3/7/5the Gauss matrixM/1 /= I /; t /(/1/)e T/1 /= /2/6/4 /1 /0 /0/0 /1 /0/0 /0 /1 /3/7/5 /; /2/6/4 /0/0/3 /= /1 /3/7/5 h/1 /0 /0 i/= /2/6/4 /1 /0 /0/0 /1 /0/; /3 /0 /1 /3/7/5 /:Th us/,M/1 PA /= /2/6/4 /1 /0 /0/0 /1 /0/; /3 /0 /1 /3/7/5 /2/6/4 /1 /2 /; /1/0 /; /2 /2/3 /5 /; /8 /3/7/5 /= /2/6/4 /1 /2 /; /1/0 /; /2 /2/0 /; /1 /; /5 /3/7/5 /;; M /; /1/1 /= /2/6/4 /1 /0 /0/0 /1 /0/3 /0 /1 /3/7/5andM/2 /= I /; t /(/2/)e T/2 /= /2/6/4 /1 /0 /0/0 /1 /0/0 /0 /1 /3/7/5 /; /2/6/4 /0/0/( /; /1/) /= /( /; /2/) /3/7/5 h/0 /1 /0 i/= /2/6/4 /1 /0 /0/0 /1 /0/0 /; /1 /2 /1 /3/7/5andM/2 M/1 PA /= /2/6/4 /1 /0 /0/0 /1 /0/0 /; /1 /2 /1 /3/7/5 /2/6/4 /1 /2 /; /1/0 /; /2 /2/0 /; /1 /; /5 /3/7/5 /= /2/6/4 /1 /2 /; /1/0 /; /2 /2/0 /0 /; /6 /3/7/5 /;; M /; /1/2 /= /2/6/4 /1 /0 /0/0 /1 /0/0 /1 /2 /1 /3/7/5 /:Consequen tly /,L /= M /; /1/1 M /; /1/2 /= /2/6/4 /1 /0 /0/0 /1 /0/3 /0 /1 /3/7/5 /2/6/4 /1 /0 /0/0 /1 /0/0 /1 /2 /1 /3/7/5 /= /2/6/4 /1 /0 /0/0 /1 /0/3 /1 /2 /1 /3/7/5 /;; U /= /2/6/4 /1 /2 /; /1/0 /; /2 /2/0 /0 /; /6 /3/7/5andPA /= LU /= /2/6/4 /1 /0 /0/0 /1 /0/3 /1 /2 /1 /3/7/5 /2/6/4 /1 /2 /; /1/0 /; /2 /2/0 /0 /; /6 /3/7/5 /:Exercise /1/./2/./3/. /Find for a certain p erm utation matrix P the LU fac/-torization of PA ifa /) A /= /"/0 /2/3 /5 /#/;; b /) A /= /"/0 /7/5 /3 /#/;; c /) A /= /2/6/4 /0 /5 /7/2 /3 /3/6 /9 /8 /3/7/5 /:/1/0/0 /2/./2 QR F actorization/2/./2/./1 Householder Re/ ectionDe/nition /2/./1/./1/. If v /2 R nand v /6/= /0 /;; then the matrix of the formH /= I /; /2 vv T v Tv /(/1/)is called the Householder matrix or Householder r e/ e ction and the v ector vis called the Householder ve ctor/.Prop osition /2/./1/./1/. The Householder matrix H is symmetric and or/-thogonal/. The Householder re/ ection re/ ects ev ery v ector x /2 R nin theh yp erplane span f v g /?/.Pr o of/.H T/= I T/; /2 /( vv T/) T v Tv /= I /; /2 vv T v Tv /= HandHH T/= H TH /=/( I /; /2 vv T v Tv /) /2/= I /; /4 vv T v Tv /+/4 v /( v Tv /) v T /( v Tv /)/( v Tv /) /= I/:T o pro v e the third part of the assertion/, w ec ho ose on the h yp erplane span f v g /?an orthogonal basis f a/1 /;;/:/:/: /;; an /; /1 g /: Hence/, v /? ai /( i /=/1/: n /; /1/) and v Tai /=/0/( i /=/1/: n /; /1/)/. Ifx /= / v /+ //1 a/1 /+ /:/:/: /+ /n /; /1 an /; /1 /;;thenH x /= H /( / v /)/+ H /( //1 a/1 /)/+ /:/:/: /+ H /( /n /; /1 an /; /1 /)/=/= / /( I /; /2 vv T v Tv /) v /+ //1 /( I /; /2 vv T v Tv /) a/1 /+ /:/:/: /+ /n /; /1 /( I /; /2 vv T v Tv /) an /; /1 /=/= / /( v /; /2 v /( v Tv /) v Tv /)/+ //1 /( a/1 /; /2 v /( v Ta/1 /) v Tv /)/+ /:/:/: /+ /n /; /1 /( an /; /1 /; /2 v /( v Tan /; /1 /) v Tv /)/=/= /; / v /+ //1 a/1 /+ /:/:/: /+ /n /; /1 an /; /1 /;;i/.e/./, the v ectors x and H x ha v e the same orthogonal pro jection on to theh yp erplane span f v g /?//1 a/1 /+ /:/:/: /+ /n /; /1 an /; /1 /;;/1/0/1 but pro jections on to the v ector v ha v e opp osite directions/. Th us H x is there/ ection of x in the h yp erplane span f v g /?/. It is signi/can t to note that theHouseholder matrix H dep ends only on the direction of Householder v ektorv and do es not dep end on the sign of the direction and length of v /.Prop osition /2/./1/./2/. If x /2 R nand v /= x /k x k/2 e/1 /;; then v ector H x /,where H is the Householder matrix denoted b y /(/1/)/, has the same directionas e/1 /, i/.e/./, the Householder re/ ection H applied to the v ector x annihilatesall but the /rst comp onen t of the v ector x /.Pr o of/. Our aim is to determine for a nonzero v ector x the Householderv ector v so that H x /2 span f e/1 g /: SinceH x /=/( I /; /2 vv T v Tv /) x /= x /; /2 v /( v Tx /) v Tv /= x /; /2 v Tx v Tv vand H x /2 span f e/1 g /;; then v /2 span f x /;; e/1 g /: By c ho osing v /= x /+ / e/1 /;; w e obtainthatv Tx /= x Tx /+ / e T/1 x /= x Tx /+ ///1 /;;v Tv /=/( x T/+ / e T/1 /)/( x /+ / e/1 /)/= x Tx /+/2 ///1 /+ / /2andH x /= x /; /2 v Tx v Tv v /= x /; /2 x Tx /+ ///1 x Tx /+/2 ///1 /+ / /2 /( x /+ / e/1 /)/=/=/( /1 /; /2 x Tx /+ ///1 x Tx /+/2 ///1 /+ / /2 /) x /; /2 / v Tx v Tv e/1 /:Cho ose / so that in the latter represen tation of H x the co e/cien t of x iszero/, i/.e/./,/1 /; /2 x Tx /+ ///1 x Tx /+/2 ///1 /+ / /2 /=/0 /,/, x Tx /+/2 ///1 /+ / /2/; /2 x Tx /; /2 ///1 /=/0 /,/, x Tx /= / /2/,k x k/2 /= / //:F or this c hoice / /= /k x k/2 w e ha v e v /= x /k x k/2 e/1 andH x /= /; /2 / v Tx v Tv e/1 /= /; /2 / x Tx / ///1 x Tx / /2 ///1 /+ x Tx e/1 /= /; / e/1 /= /k x k/2 e/1 /:Example /2/./1/./1 Let x /=/[/2 /6 /; /3/] T/: Find the Householder v ector v andhence to it the Householder transformation that annihilates the t w o last co or/-dinates of the v ector x /. By Prop osition /2/./1/./1 w e compute v /= x /k x k/2 e/1 /=/1/0/2 /[/2 /6 /; /3/] T/ /7 e/1 /: Cho ose the sign plus for co e/cien t of e/1and w e obtain v /=/[/9 /6 /; /3/] T/: Find the Householder matrix H that dep endsonly on direction of v /,H /= I /; /2 v Tv vv T/= I /; /2 /1/4 /2/6/4 /3/2/; /1 /3/7/5 h/3 /2 /; /1 i/=/= I /; /1 /7 /2/6/4 /9 /6 /; /3/6 /4 /; /2/; /3 /; /2 /1 /3/7/5 /= /1 /7 /2/6/4 /; /2 /; /6 /3/; /6 /3 /2/3 /2 /6 /3/7/5 /:Chec k/,H x /= /1 /7 /2/6/4 /; /2 /; /6 /3/; /6 /3 /2/3 /2 /6 /3/7/5 /2/6/4 /2/6/; /3 /3/7/5 /= /2/6/4 /; /7/0/0 /3/7/5 /:Exercise /2/././1/./1/./* Find the Householder matrix H suc h that H x /2span f e/1 g /, where x /= h/; /3 /1 /; /5 /1 iT/:Let Qi /2 R n / n/( i /=/1 /: r /) be the Householder matrices/. Consider thepro duct of these matricesQ /= Q/1 /
/
/
Qr /;;whereQj /= I /; /j v /( j /)v /( j /) Tand eac h v /( j /)has the formv /( j /)/= h/0 /
/
/
/0 /1 / /( j /)j /+/1 /
/
/
/ /( j /)n iT/:j /; /1 zerosThe matrix Q can be written in the formQ /= I /+ WY T/;; /(/2/)where W and Y are n / r /; matrices/. The answ er to the question ho w to /ndrepresen tation /(/2/) is giv en b y the follo wing prop osition/.Prop osition /2/./1/./3/. Supp ose Q /= I /+ WY T/2 R n / nis an orthogonalmatrix with W /;;Y /2 R n / j/: If H /= I /; / vv T/;; where v /2 R nand z /= /; /Q v /;;thenQ/+ /= QH /= I /+ W/+ Y/+ T/;;/1/0/3 where W/+ /=/[ W z /]a n d Y/+ /=/[ Y v /] /;; and consequen tly /, W/+ /, Y/+ /2 R n / /( j /+/1/)/:Pr o of/. SinceQH /=/( I /+ WY T/)/( I /; / vv T/)/= I /+ WY T/; / /( I /+ WY T/) vv T/=/= I /+ WY T/; /Q vv T/= I /+ WY T/; zv TandI /+/[ W z /] /"Y Tv T /#/= I /+ hWY T/+ zv T i/= I /+ WY T/+ zv T/;;then QH /= I /+ W/+ Y/+ T/;; and the assertion of the prop osition holds/./2/./2/./2 Giv ens RotationsThe Householder re/ ection is e/ectiv e for in truducing zeros if there are/"man y/" comp onen ts to be annihilated/. If it is necessary to n ullify only onecomp onen t or sometimes a couple of comp onen ts/, then usually the Giv ensmetho d is used /. The Givens r otation is realized b y an n / n /; matrixG /( i/;; k /;; / /) /= /2/6/6 /6 /6 /6 /6/6/6 /6 /6 /6/6/6/4 /1 /
/
/
/0 /
/
/
/0 /
/
/
/0/././. /././. /././. /././. /./././0 /
/
/
c /
/
/
s /
/
/
/0/././. /././. /././. /././. /./././0 /
/
/
/; s /
/
/
c /
/
/
/0/././. /././. /././. /././. /./././0 /
/
/
/0 /
/
/
/0 /
/
/
/1 /3/7/7 /7 /7 /7 /7/7/7 /7 /7 /7/7/7/5 iki kwhere c /=c o s / and s /=s i n / /. This matrix G /( i/;; k /;; / /) is eviden tly orthogonal/.If x /2 R nand y /= G /( i/;; k /;; / /) Tx /;; then/j /= /8/>/</>/: c/i /; s/k /;; j /= i /,s/i /+ c/k /;; j /= k/;;/j /;; j /6/= i/;; k /:/1/0/4 By settingc /= /i q / /2i /+ / /2k /^ s /= /; /k q / /2i /+ / /2k /;;w e get /k /=/0 /:Example /2/./2/./1/. Consider the annihilation of the last comp onen t of thev ector x /=/[/2 /6 /; /3/] Tgiv en in example /2/./1/./1 with Giv ens rotations/. Find thev alues of c and s /:c /= //2 q / /2/2 /+ / /2/3 /= /6 p /4/5 /= /2 p /5 /5 /;;s /= /; //3 q / /2/2 /+ / /2/3 /= /3 p /4/5 /= p /5 /5 /:Chec k/2/6/6/4 /1 /0 /0/0 /2 p /5 /5 /; p /5 /5/0 p /5 /5 /2 p /5 /5 /3/7/7/5 /2/6/4 /2/6/; /3 /3/7/5 /= /2/6/4 /2/1/5 p /5 /5/0 /3/7/5 /= /2/6/4 /2/3 p /5/0 /3/7/5 /:/2/./2/./3 Householder QR F actorizationApply the Householder re/ ection to the matrix A /2 R m / n/( m / n /) toobtain the QR factorization/.Example /2/./3/./1/. Supp ose A /2 R /5 / /4and assume that the Householdermatrices H/1 and H/2 ha v e b een computed so thatH/2 H/1 A /= /2/6/6 /6 /6 /6/6/4 / / / //0 / / //0 /0 / //0 /0 / //0 /0 / / /3/7/7 /7 /7 /7/7/5 /:Concen trating on the highligh ted v ector /2/6/4 / / / /3/7/5 /,w e determine a Householder/1/0/5 matrix fH/3 suc h thatfH/3 /2/6/4 / / / /3/7/5 /= /2/6/4 //0/0 /3/7/5Cho osing H/3 /= diag /( I/2 /;; fH/3 /) /;; w e getH/3 H/2 H/1 A /= /2/6/6 /6 /6/6/6/4 / / / //0 / / //0 /0 / //0 /0 /0 / /0 /0 /0 / /3/7/7 /7 /7/7/7/5 /:Next consider the highligh ted v ector /"/ / /#and determine fH/4 suc h thatfH/4 /"/ / /#/= /"//0 /#/:Cho osing H/4 /= diag /( I/3 /;; fH/4 /) /;; w e ha v eH/4 H/3 H/2 H/1 A /= /2/6/6 /6 /6 /6/6/4 / / / //0 / / //0 /0 / //0 /0 /0 //0 /0 /0 /0 /3/7/7 /7 /7 /7/7/5 /= R/:By setting Q /= H/1 H/2 H/3 H/4 /;; w e obtain QR /= H/1 H/2 H/3 H/4 H/4 H/3 H/2 H/1 A /= A/:Prop osition /2/./3/./1/. If A /2 R m / n/( m / n /) /;; then there exist Householdermatrices Hi suc h thatQ /= /(H/1 /
/
/
Hn /;; kui m/>n /;;H/1 /
/
/
Hn /; /1 /;; kui m /= n /;;R /= /(Hn /
/
/
H/1 A/;; kui m/>n /;;H/1 /
/
/
Hn /; /1 A/;; kui m /= nandA /= QR /;;where Q /2 R m / mis orthogonal and R /2 R m / nis upp er triangular/./1/0/6 Example /2/./3/./2/. Find the Householder QR factorization forA /= /2/6/4 /2 /0 /1/6 /2 /0/; /3 /; /1 /; /1 /3/7/5 /:In example /2/./1/./1 the Householder matrix for the transformation of the /rstcolumn v ector /[/2 /6 /; /3/] Tof A has b een found/:H/1 /= /1 /7 /2/6/4 /; /2 /; /6 /3/; /6 /3 /2/3 /2 /6 /3/7/5 /:ThenH/1 A /= /1 /7 /2/6/4 /; /2 /; /6 /3/; /6 /3 /2/3 /2 /6 /3/7/5 /2/6/4 /2 /0 /1/6 /2 /0/; /3 /; /1 /; /1 /3/7/5 /= /1 /7 /2/6/4 /; /4/9 /; /1/5 /; /5/0 /4 /; /8/0 /; /2 /; /3 /3/7/5 /:T o /nd fH/2 /;; w e compute the Householder v ectorv /= /"/4/; /2 /#/; p /2/0 /"/1/0 /#/= /"/4 /; /2 p /5/; /2 /#/:HencefH/2 /= I /; /2 vv T v Tv /= /
/
/
/= p /5 /5 /"/2 /; /1/; /1 /; /2 /#andH/2 /= diag /( I/1 /;; fH/2 /)/= /2/6/6/4 /1 /0 /0/0 /2 p /5 /5 /; p /5 /5/0 /; p /5 /5 /; /2 p /5 /5 /3/7/7/5and alsoR /= H/2 H/1 A /= /1 /7 /2/6/6/4 /1 /0 /0/0 /2 p /5 /5 /; p /5 /5/0 /; p /5 /5 /; /2 p /5 /5 /3/7/7/5 /2/6/4 /; /4/9 /; /1/5 /; /5/0 /4 /; /8/0 /; /2 /; /3 /3/7/5 /=/= /2/6/6/4 /; /7 /; /1/5 /7 /; /5 /7/0 /2 p /5 /7 /; /1/3 p /5 /3/5/0 /0 /2 p /5 /5 /3/7/7/5 /:/1/0/7 Find also the orthogonal matrixQ /= H/1 H/2 /= /1 /7 /2/6/6/4 /1 /0 /0/0 /2 p /5 /5 /; p /5 /5/0 /; p /5 /5 /; /2 p /5 /5 /3/7/7/5 /2/6/4 /; /2 /; /6 /3/; /6 /3 /2/3 /2 /6 /3/7/5 /=/= p /5 /3/5 /2/6/4 /; /2 p /5 /; /1/5 /0/; /6 p /5 /4 /; /7/3 p /5 /; /2 /; /1/4 /3/7/5and c hec k the resultQR /= p /5 /3/5 /2/6/4 /; /2 p /5 /; /1/5 /0/; /6 p /5 /4 /; /7/3 p /5 /; /2 /; /1/4 /3/7/5 /2/6/6/4 /; /7 /; /1/5 /7 /; /5 /7/0 /2 p /5 /7 /; /1/3 p /5 /3/5/0 /0 /2 p /5 /5 /3/7/7/5 /= A/:Example /2/./3/./3 //. Find the Householder QR factorization ofA /= /2/6/4 /1 /1/2 /3/2 /1 /3/7/5 /:The v ector that has to be transformed is x /= h/1 /2 /2 iT/;;/;; where k x k/2 /=p /1 /2/+/2 /2/+/2 /2/=/3 /: Construct the v ectorv /= x /k x k/2 e/1 /= /2/6/4 /1/2/2 /3/7/5 / /3 e/1 /:Cho ose a min us sign for the co e/cien t of e/1 and tak e in to accoun t that Hdep ends only on the direction of v /:v /= h/; /2 /2 /2 iT/ h/; /1 /1 /1 iT/:Find the Householder matrixH/1 /= I /; /2 v Tv vv T/= I /; /2 /( /; /1/)/( /; /1 /)/+/1 /
/1/+/1 /
/1 /2/6/4 /; /1/1/1 /3/7/5 h/; /1 /1 /1 i/=/1/0/8 /= I /; /2 /3 /2/6/4 /1 /; /1 /; /1/; /1 /1 /1/; /1 /1 /1 /3/7/5 /= /1 /3 /2/6/4 /1 /2 /2/2 /1 /; /2/2 /; /2 /1 /3/7/5 /:V erify that H/1 annihilates all the elemen ts of the /rst column of A but the/rst one/.H/1 A /= /1 /3 /2/6/4 /1 /2 /2/2 /1 /; /2/2 /; /2 /1 /3/7/5 /2/6/4 /1 /1/2 /3/2 /1 /3/7/5 /= /1 /3 /2/6/4 /9 /9/0 /3/0 /; /3 /3/7/5 /= /2/6/4 /3 /3/0 /1/0 /; /1 /3/7/5 /:F urther w e transform the v ector x /= h/1 /; /1 iT/;; where k x k/2 /= p /2 /: Find theHouseholder v ector according to xv /= x /k x k/2 e/1 /= /"/; /1/1 /#/ p /2 e/1 /:Cho ose am i n us sign for co e/cien t of e/1 /:v /= /"/; /1 /; p /2/1 /#/:Using this v ector w e obtain the Householder matrixfH/2 /= I /; /2 v Tv vv T/= I /; /2 /( /; /1 /; p /2 /) /2/+/1 /"/; /1 /; p /2/1 /#h/; /1 /; p /2 /1 i/=/= I /; /2 /2/(/2 /; p /2/) /"/3 /; /2 p /2 p /2 /; /1p /2 /; /1 /1 /#/= /1 /2 /; p /2 /"/; /1/+ p /2 /; p /2/+/1/; p /2/+/1 /; p /2/+/1 /#/=/= p /2 /; /1 /2 /; p /2 /"/1 /; /1/; /1 /; /1 /#/= p /2 /2 /"/1 /; /1/; /1 /; /1 /#and /nd thatH/2 /= diag /( I/1 /;; fH/2 /)/= /2/6/4 /1 /0 /0/0 p /2 /= /2 /; p /2 /= /2/0 /; p /2 /= /2 /; p /2 /= /2 /3/7/5 /:Th us/,R /= H/2 H/1 A /= /2/6/4 /1 /0 /0/0 /1 /2 p /2 /; /1 /2 p /2/0 /; /1 /2 p /2 /; /1 /2 p /2 /3/7/5 /2/6/4 /3 /3/0 /1/0 /; /1 /3/7/5 /= /2/6/4 /3 /3/0 p /2/0 /0 /3/7/5/1/0/9 andQ /= H/1 H/2 /= /1 /3 /2/6/4 /1 /2 /2/2 /1 /; /2/2 /; /2 /1 /3/7/5 /2/6/4 /1 /0 /0/0 /1 /2 p /2 /; /1 /2 p /2/0 /; /1 /2 p /2 /; /1 /2 p /2 /3/7/5 /= /2/6/4 /1 /3 /0 /; /2 /3 p /2/2 /3 /1 /2 p /2 /1 /6 p /2/2 /3 /; /1 /2 p /2 /1 /6 p /2 /3/7/5 /:Let us c hec k the result/:QR /= /2/6/4 /1 /3 /0 /; /2 /3 p /2/2 /3 /1 /2 p /2 /1 /6 p /2/2 /3 /; /1 /2 p /2 /1 /6 p /2 /3/7/5 /2/6/4 /3 /3/0 p /2/0 /0 /3/7/5 /= /2/6/4 /1 /1/2 /3/2 /1 /3/7/5 /= A/:Exercise /2/./3/./1/. /Find the QR factorization of A ifa /) A /= /2/6/4 /0 /0/1 /3/0 /2 /3/7/5 /;; b /) A /= /"/5 /9/1/2 /7 /#/;; c /) A /= /2/6/4 /3 /3 /0/3 /5 /0/0 /0 /6 /3/7/5 /;;d /) A /= /2/6/6 /6/4 /1 /0 /0 /1/3 /1 /0 /0/5 /1 /1 /0/1 /0 /0 /1 /3/7/7 /7/5 /:/2/./2/./4 Giv ens QR F actorizationNext w e consider ho w to use the Giv ens rotations to compute the QRfactorization of a giv en matrix/.Example /2/./4/./1/. Consider for A /2 R /4 / /3the idea of the Givens QRfactorization /:A /= /2/6/6/6/4 / / // / // / // / / /3/7/7/7/5 G T/1 /(/3 /;; /4/)/; /! /2/6/6/6/4 / / // / // / //0 / / /3/7/7/7/5 G T/2 /(/2 /;; /3/)/; /! /2/6/6/6/4 / / // / //0 / //0 / / /3/7/7/7/5 G T/3 /(/1 /;; /2/)/; /!/2/6/6 /6/4 / / //0 / //0 / //0 / / /3/7/7 /7/5 G T/4 /(/3 /;; /4/)/; /! /2/6/6 /6/4 / / //0 / //0 / //0 /0 / /3/7/7 /7/5 G T/5 /(/2 /;; /3/)/; /! /2/6/6 /6/4 / / //0 / //0 /0 //0 /0 / /3/7/7 /7/5 G T/6 /(/3 /;; /4/)/; /!/1/1/0 /2/6/6 /6/4 / / //0 / //0 /0 //0 /0 /0 /3/7/7 /7/5 /= RThe orthogonal matrix has the form/:Q /= G/1 /(/3 /;; /4/) G/2 /(/2 /;; /3/) G/3 /(/1 /;; /2/) G/4 /(/3 /;; /4/) G/5 /(/2 /;; /3/) G/6 /(/3 /;; /4/) /:Example /2/./4/./2/. /Find the Giv ens QR factorization ofA /= /2/6/4 /2 /0 /1/6 /2 /0/; /3 /1 /; /1 /3/7/5 /:Let us annihilate the elemen t A /(/3 /;; /1/) of A/: F or this w e construct the Giv ensmatrix G/1 /(/2 /;; /3/) /: Find the v alues c and s /:c /= /6 q /6 /2/+/( /; /3/) /2 /= /6 p /4/5 /= /2 p /5 /5 /;; s /= /3 q /6 /2/+/( /; /3/) /2 /= /3 p /4/5 /= p /5 /5 /:Th us/, w e ha v eG/1 /(/2 /;; /3/) /= /2/6/4 /1 /0 /0/0 /2 /5 p /5 /1 /5 p /5/0 /; /1 /5 p /5 /2 /5 p /5 /3/7/5andA /(/1/)/= G T/1 /(/2 /;; /3/) A /= /2/6/4 /1 /0 /0/0 /2 /5 p /5 /; /1 /5 p /5/0 /1 /5 p /5 /2 /5 p /5 /3/7/5 /2/6/4 /2 /0 /1/6 /2 /0/; /3 /1 /; /1 /3/7/5 /=/= /2/6/4 /2 /0 /1/3 p /5 /3 /5 p /5 /1 /5 p /5/0 /4 /5 p /5 /; /2 /5 p /5 /3/7/5 /:F or the annihilation of the elemen t A /(/1/)/(/2 /;; /1/) of A /(/1/)w e construct the Giv ensmatrix G/2 /(/1 /;; /2/) /: Find the v alues c and s /:c /= /2 q /2 /2/+/( /3 p /5/) /2 /= /2 /7 /;; s /= /; /3 p /5 q /2 /2/+/( /3 p /5/) /2 /= /; /3 p /5 /7 /:/1/1/1 Th us/,G/2 /(/1 /;; /2/) /= /2/6/4 /2 /7 /; /3 /7 p /5 /0/3 /7 p /5 /2 /7 /0/0 /0 /1 /3/7/5andA /(/2/)/= G T/2 /(/1 /;; /2/) A /(/1/)/= /2/6/4 /2 /7 /3 /7 p /5 /0/; /3 /7 p /5 /2 /7 /0/0 /0 /1 /3/7/5 /2/6/4 /2 /0 /1/3 p /5 /3 /5 p /5 /1 /5 p /5/0 /4 /5 p /5 /; /2 /5 p /5 /3/7/5 /=/= /2/6/4 /7 /9 /7 /5 /7/0 /6 /3/5 p /5 /; /1/3 /3/5 p /5/0 /4 /5 p /5 /; /2 /5 p /5 /3/7/5 /:T o annihilate the elemen t A /(/2/)/(/3 /;; /2/) of A /(/2/)w e construct the Giv ens matrixG/3 /(/2 /;; /3/) /: Find the v alues of c and s /:c /= /6 /3/5 p /5 r //6 /3/5 p /5 //2/+ //4 /5 p /5 //2 /= /6 /3/5 p /5 /2 /7 p /4/1 /= /3 /2/0/5 p /2/0/5ands /= /; /4 /5 p /5 r //6 /3/5 p /5 //2/+ //4 /5 p /5 //2 /= /; /1/4 /2/0/5 p /2/0/5 /:Th us/,G/3 /(/2 /;; /3/) /= /2/6/4 /1 /0 /0/0 /3 /2/0/5 p /2/0/5 /; /1/4 /2/0/5 p /2/0/5/0 /1/4 /2/0/5 p /2/0/5 /3 /2/0/5 p /2/0/5 /3/7/5andR /= G T/3 /(/2 /;; /3/) A /(/2/)/= /2/6/4 /1 /0 /0/0 /3 /2/0/5 p /2/0/5 /1/4 /2/0/5 p /2/0/5/0 /; /1/4 /2/0/5 p /2/0/5 /3 /2/0/5 p /2/0/5 /3/7/5 /2/6/4 /7 /9 /7 /5 /7/0 /6 /3/5 p /5 /; /1/3 /3/5 p /5/0 /4 /5 p /5 /; /2 /5 p /5 /3/7/5 /=/:/= /2/6/4 /7 /9 /7 /5 /7/0 /2 /7 p /4/1 /; /4/7 /2/8/7 p /4/1/0 /0 /4 /4/1 p /4/1 /3/7/5/1/1/2 andQ /= G/1 /(/2 /;; /3/) G/2 /(/1 /;; /2/) G/3 /(/2 /;; /3/) /=/= /2/6/4 /1 /0 /0/0 /2 /5 p /5 /1 /5 p /5/0 /; /1 /5 p /5 /2 /5 p /5 /3/7/5 /2/6/6/4 /2 /7 /; /3 p /5 /7 /0/3 p /5 /7 /2 /7 /0/0 /0 /1 /3/7/7/5 /2/6/4 /1 /0 /0/0 /3 /2/0/5 p /2/0/5 /; /1/4 /2/0/5 p /2/0/5/0 /1/4 /2/0/5 p /2/0/5 /3 /2/0/5 p /2/0/5 /3/7/5 /=/= /2/6/4 /2 /7 /; /9 /2/8/7 p /4/1 /6 /4/1 p /4/1/6 /7 /2/2 /2/8/7 p /4/1 /; /1 /4/1 p /4/1/; /3 /7 /3/8 /2/8/7 p /4/1 /2 /4/1 p /4/1 /3/7/5 /:Let us c hec k/:QR /= /2/6/4 /2 /7 /; /9 /2/8/7 p /4/1 /6 /4/1 p /4/1/6 /7 /2/2 /2/8/7 p /4/1 /; /1 /4/1 p /4/1/; /3 /7 /3/8 /2/8/7 p /4/1 /2 /4/1 p /4/1 /3/7/5 /2/6/4 /7 /9 /7 /5 /7/0 /2 /7 p /4/1 /; /4/7 /2/8/7 p /4/1/0 /0 /4 /4/1 p /4/1 /3/7/5 /=/= /2/6/4 /2 /0 /1/6 /2 /0/; /3 /1 /; /1 /3/7/5 /= A/:/:Exercise /2/./4/./1/. Find the Giv ens QR factorization of the matrix A inexample /2/./3/./2/.Exercise /2/./4/./2/. /Find the Giv ens QR factorization of A ifa /) A /= /2/6/4 /; /1/2 /1/4 /0/3 /3 /3/7/5 /;; b /) A /= /"/; /6 /2/8 /4 /#/;; c /) A /= /2/6/4 /1/2 /; /3 /1/; /3 /1 /2/4 /; /4 /3 /; /1 /3/7/5 /:/2/./2/./5 Main Theorem of QR F actorizationProp osition /2/./5/./1/. If A /= /[ a/1 /
/
/
an /] /2 R m / n/( m / n /) with linearlyindep enden t column v ectors ai /( i /=/1 /: n /) can be factored in to A /= QR /;;where Q /=/[ q/1 /
/
/
qm /] /2 R m / mand R /2 R m / n/;; thenspan f a/1 /;;/:/:/: /;; ak g /= span f q/1 /;;/:/:/: /;; qk g /( k /=/1 /: n /)/. /(/3/)/1/1/3 In particular/, ifQ/1 /= Q /(/1 /: m/;; /1/: n /) /;; Q/2 /= Q /(/1 /: m/;; n /+/1 /: m /) /;; R/1 /= R /(/1 /: n/;; /1/: n /) /;;thenR /( A /)/= R /( Q/1 /) /(/4/)R /( A /) /?/= R /( Q/2 /) /(/5/)andA /= Q/1 R/1 /;; /(/6/)Pr o of/. If A /= QR /;; thenaik /= mXj /=/1 qij rjk rjk /=/0/=j/> k kXj /=/1 qij rjk /( i /=/1 /: m/;; k /=/1 /: n /)orak /= kXj /=/1 rjk qj /( k /=/1 /: n /) /:Th us/, ak /2 span f q/1 /;;/:/:/: /;; qk g and span f a/1 /;;/:/:/: ak g/ span f q/1 /;;/:/:/: /;; qk g /: Sincera n k /( A /)/= n/;; then ra n k /( span f a/1 /;;/:/:/: ak g /)/= k/;; and relation /(/3/) holds/. Rela/-tion /(/3/) for k /= n yields relation /(/4/)/, and this yields /(/5/)/. F romaik /= mXj /=/1 qij rjk /= nXj /=/1 qij rjkresults assertion /(/6/)/. /2/2/./3 Singular V alue Decomp osition/2/./3/./1 Existence of Singular V alue Decomp ositionProp osition /3/./1/./1/. If V/1 /2 R n / r/( r /< n /) has orthonormal columns/,then there exists V/2 /2 R n / /( n /; r /)suc h that V /=/[ V/1 V/2 /] is orthogonal/, wherethe orthogonal complemen t R /( V/1 /) /?of the span of column v ectors of thematrix V/1 is equal to the span R /( V/2 /)o ft h e column v ectors of the matrix V/2/;; i/.e/./, R /( V/1 /) /?/= R /( V/2 /) /:/1/1/4 Pr o of is based on the Gram/-Sc hmidt orthogonalization/. /2Prop osition /3/./1/./2/. If x /2 R nand Q /2 R m / nhas orthonormal columns/,then k Q x k/2 /= k x k/2 /:Pr o of/. If Q /2 R m / nhas orthonormal columns/, then Q TQ /= In andk Q x k /2/2 /=/( Q x /) TQ x /= x TQ TQ x /= x Tx /= k x k /2/2 /: /2Prop osition /3/./1/./3/. Let A /2 R m / n/: If Q /2 R m / mand Z /2 R n / nareorthogonal/, thenk QAZ kF /= k A kFandk QAZ k/2 /= k A k/2 /: /(/1/)Pr ove relation /(/1/)/:k QAZ k/2 /= maxk x k/2 /=/1 k QAZ x k/2 /= maxk x k/2 /=/1 k QA /( Z x /) k/2 /=/=m a xk z k/2 /=/1 k QA z k/2 /= maxk z k/2 /=/1 k Q /( A z /) k/2 /= maxk z k/2 /=/1 k A z k/2 /= k A k/2 /: /2Prop osition /3/./1/./4 /(existenc e the or em of the singular value de c omp osi/-tion/)/. If A /2 R m / n/;; then there exist orthogonal matricesU /=/[ u/1 /
/
/
um /] /2 R m / mandV /=/[ v/1 /
/
/
vn /] /2 R n / n/;;suc h thatU TAV /=//= diag /( //1 /;;/:/:/: /;;/p /) /2 R m / n/( p /= min f m/;; n g /) /(/2/)with//1 / //2 / /:/:/: / /p / /0 /:Pr o of/. By the de/nition of the matrix /2/-norm there exist v ectors x /2 R nand y /2 R msuc h that A x /= / y /;; where k x k/2 /= k y k/2 /= /1 and / /= k A k/2 /:By Prop osition /3/./1/./1/, there exist matrices V/2 /2 R n / /( n /; /1/)and U/2 /2 R m / /( m /; /1/)/1/1/5 suc h that V /= /[ x V/2 /] and U /= /[ y U/2 /] are orthogonal/. Using this notation/,w e obtainU TAV /= /"y TU T/2 /#A hx V/2 i/= /"y TU T/2 /#hA x AV/2 i/=/= /"y TU T/2 /#h/ y AV/2 i/= /"/ y Ty y TAV/2/U T/2 y U T/2 AV/2 /#/=/= /"/ w T/0 B /#/= A/1with w /= V T/2 A Ty and B /= U T/2 AV/2 /: SinceA/1 /"/w /#/= /"/ w T/0 B /#/"/w /#/= /"/ /2/+ w TwB w /#/;;then/ / / / / A/1 /"/w /# / / / / / /2/2 / /( / /2/+ w Tw /) /2/:On the other hand/,/ / / / / A/1 /"/w /# / / / / / /2/2 /k A/1 k /2/2 / / / / / /"/w /# / / / / / /2/2 /= k A/1 k /2/2 /( / /2/+ w Tw /) /;;and therefore/,k A/1 k /2/2 / / /2/+ w Tw /= k A k /2/2 /+ w Tw /:By Prop osition /3/./1/./3/, w e /nd that k A/1 k /2/2 /= k A k /2/2 /: Consequen tly /, w Tw /=/0and w /= /0 /: W e obtainU TAV /= /"/ /0 T/0 B /#orA /= U /"/ /0 T/0 B /#V TandA TA /= V /"/ /0 T/0 B T /#U TU /"/ /0 T/0 B /#V T/= V /"/ /2/0/0 B TB /#V T/:/1/1/6 Th us/, the matrices A TA and /"/ /2/0 T/0 B TB /#are similar/, and they ha v e thesame eigen v alues/. Consequen tly /,/ /( A TA /)/= f / /2g/[ / /( B TB /) /;;where / /2as k A k /2/2 is the greatest eigen v alue of A TA /. Note that since A TAis symmetric then all eigen v alues of A TA are non/-negativ e/. The Reasoningused for the matrix A will b e used in the next step for the matrix B etc/. So/,on the main diagonal of / there are the square ro ots of the eigen v alues ofA TA /, more exactly /, the /rst p /=m i n f m/;; n g of them in descending order/. /2De/nition /3/./1/./1/. The relation in form /(/2/) is called the singular valuede c omp osition of the matrix A /2 R m / n/: The elemen ts /i /( i /= /1 /: min f m/;; n g /)on the main diagonal of / are called the singular values of the matrix A /./2/./3/./2 Prop erties of Singular V alue Decomp ositionRelation /(/2/) yields the relationsAV /= U / /(/3/)andA TU /= V / T/: /(/4/)Prop osition /3/./2/./1/. If A /2 R m / n/;;A /= U / V T/;; U /=/[ u/1 /
/
/
um /] /2 R m / mandV /=/[ v/1 /
/
/
vn /] /2 R n / n/;; then for eac h i /= /1 /: min f m/;; n g the follo wing holdsA vi /= /i ui /;; /(/5/)A Tui /= /i vi /;; /(/6/)k A kF /= / /2/1 /+ /:/:/: /+ / /2p /( p /= min f m/;; n g /) /;;k A k/2 /= //1andminx /6/= /0 k A x k/2 k x k/2 /= /n /( m / n /) /:/1/1/7 Pr o of/. Supp ose n/> m /: Consider relation /(/3/) whic h can b e written in theformA /[ v/1 /
/
/
vn /]/= /[ u/1 /
/
/
um /] /2/6/6/6 /6/4 //1 /0 /
/
/
/0 /0/0 //2 /
/
/
/0 /0/./. /. /./. /. /././. /./. /. /./. /./0 /0 /
/
/
/m /0 /3/7/7/7 /7/5or/[ A v/1 /
/
/
A vn /]/= h//1 u/1 /
/
/
/m um /0 i/:The latter is /(/5/) for the elemen ts in the m /rst columns of the matrix/. Con/-sider relation /(/4/) that can b e written in the formA T/[ u/1 /
/
/
um /]/= /[ v/1 /
/
/
vn /] /2/6/6/6/6/6 /6 /6/4 //1 /0 /
/
/
/0/0 //2 /
/
/
/0/./. /. /./. /. /././. /./. /./0 /0 /
/
/
/m/0 /0 /
/
/
/0 /3/7/7/7/7/7 /7 /7/5or/[ A Tu/1 /
/
/
A Tum /]/= h//1 v/1 /
/
/
/m vm i/;;whic h represen ts relation /(/6/) b y elemen ts/. W e note that /"/0/" denotes alsocertain blo c ks consisting of zeros/. /2Prop osition /3/./2/./2/. If the singular v alues in the singular v alue decom/-p osition /(/2/) of A /2 R m / nsatisfy the inequalities//1 / /:/:/: / /r />/r /+/1 /= /:/:/: /= /p /=/0 /;;then/1/. span f u/1 /;;/:/:/: /;; ur g /= R /( A /)/;;/2/. span f v/1 /;;/:/:/: /;; vr g /= R /( A T/)/;;/3/. span f ur /+/1 /;;/:/:/: /;; um g /= N /( A T/)/;;/4/. span f vr /+/1 /;;/:/:/: /;; vn g /= N /( A /)/;;/5/. ra n k /( A /)/= r /;;/1/1/8 /6/. the singular v alues of A are equal to the semi/-axes of the h yp erellipsoidE /= f A x /: jj x jj /=/1 g /;;/7/. A /= Pri /=/1 /i ui v Ti /:Pr ove the /rst of these prop erties/. Consider the relation A /= U / V T/.Since/[/ V T/]jk /= nXs /=/1 /js v Tsk /= /(/j vkj /;; if j /=/1 /: r /;;/0 /;; kui j /= r /+/1 /: m/;;thenaik /=/[ U / V T/]ik /= mXj /=/1 uij /[/ V T/]jk /= rXj /=/1 uij /j vkjorak /= rXj /=/1 /j vkj uj /:Th us/,ak /2 span f u/1 /;;/:/:/: /;; ur g /( k /=/1 /: n /) /) span f u/1 /;;/:/:/: /;; ur g /= R /( A /) /: /2Prop osition /3/./2/./3/. If A /2 R m / nand A /= U / V Tis a singular v aluedecomp osition of the matrix A /, then the column/-v ectors of U /2 R m / marethe normed eigen v ectors of AA Tand the column/-v ectors of V /2 R n / nare thenormed eigen v ectors of A TA /. Singular v alues of the matrix A can be foundas square ro ots of the eigen v alues of A TA or AA T/.Pr o of/. Pro ceeding from the singular v alue decomp osition of the matrixA w e will /nd expressions of AA Tand A TA /:AA T/= U / V TV / TU T/= U /(// T/) U T/(/7/)andA TA /= V / TU TU / V T/= V / T/ V T/: /(/8/)Since the matrices // Tand / T/ are diagonal matrices/, the orthogonal matri/-ces U and V in the expressions /(/7/) and /(/8/) m ust b e formed b y the eigen v ectorsof the matrices AA Tand A TA resp ectiv ely /. /2/1/1/9 /2/./3/./3 Algorithm of Singular V alue Decomp ositionAlgorithm /3/./3/./1/. T o /nd the singular v alue decomp osition of the matrixA /2 R m / none has to/:I Find the eigen v alues of the matrix A TA and arrange them in descendingorder/.I I Find the n um be r of nonzero eigen v alues of the matrix A TA /.I I I Find the orthogonal eigen v ectors of the matrix A TA corresp onding to theeigen v alues/, and arrange them in the same order to form the column/-v ectorsof the matrix V /2 R n / n/.IV F orm a diagonal matrix / /2 R m / nplacing on the leading diagonal thesquare ro ots /i /= p /i of p /= min f m/;; n g /rst eigen v alues of the matrix A TAobtained in I in descending order/.V Find the /rst column/-v ectors of the matrix U /2 R m / m/:ui /= / /; /1i A vi /( i /=/1 /: r /) /: /(/9/)VI Add to the matrix U the rest of m /; r v ectors using the Gram/-Sc hmidtorthogonalization pro cess/. /2Example /3/./3/./1/. Let us /nd the singular v alue decomp osition of thematrixA /= /2/6/4 /1 /1/0 /1/1 /0 /3/7/5 /2 R /3 / /2/:I Find the eigen v alues of the matrix A TA /= /"/2 /1/1 /2 /#/://1 /=/3 /;; //2 /=/1 /:I I Find the n um be r of nonzero eigen v alues of the matrix A TA /: r /=/2 /.III Find the orthonormal eigen v ectors of the matrix A TA corresp onding tothe eigen v alues //1 and //2 /:v/1 /= /"p /2 /= /2p /2 /= /2 /#and v/2 /= /"p /2 /= /2/; p /2 /= /2 /#forming a matrixV /= hv/1 v/2 i/= /"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/2 R /2 / /2/:/1/2/0 IV Find the singular v alue matrix / /2 R /3 / /2/://= /2/6/4 p /3 /0/0 p /1/0 /0 /3/7/5 /= /2/6/4 p /3 /0/0 /1/0 /0 /3/7/5 /;;on the leading diagonal of whic h are the square ro ots of the eigen v alues of thematrix A TA /(in descending order/) and the rest of the en tries of the matrix/ are zeros/.V Find the /rst t w o column/-v ectors of the matrix U /2 R /3 / /3using the form ula/(/9/)u/1 /= / /; /1/1 A v/1 /= p /3 /3 /2/6/4 /1 /1/0 /1/1 /0 /3/7/5 /"p /2 /= /2p /2 /= /2 /#/= /2/6/4 p /6 /= /3p /6 /= /6p /6 /= /6 /3/7/5andu/2 /= / /; /1/2 A v/2 /= /2/6/4 /1 /1/0 /1/1 /0 /3/7/5 /"p /2 /= /2/; p /2 /= /2 /#/= /2/6/4 /0/; p /2 /= /2p /2 /= /2 /3/7/5 /:VI T o /nd the v ector u/3 w e shall /rst /nd/, applying the Gram/-Sc hmitdpro cess/, a v ector / u/3 p erp endicular to u/1 and u/2 /:/ u/3 /= e/1 /; /( u T/1 e/1 /) u/1 /; /( u T/1 e/2 /) u/2 /= h/1 /= /3 /; /1 /= /3 /; /1 /= /3 iT/:Norming the v ector / u/3 /;; w e getu/3 /= /2/6/4 p /3 /= /3/; p /3 /= /3/; p /3 /= /3 /3/7/5 /:HenceU /= hu/1 u/2 u/3 i/= /2/6/4 p /6 /= /3 /0 p /3 /= /3p /6 /= /6 p /2 /= /2 /; p /3 /= /3p /6 /= /6 /; p /2 /= /2 /; p /3 /= /3 /3/7/5and the singular v alue decomp osition of the matrix A isA /= /2/6/4 p /6 /= /3 /0 p /3 /= /3p /6 /= /6 /; p /2 /= /2 /; p /3 /= /3p /6 /= /6 p /2 /= /2 /; p /3 /= /3 /3/7/5 /2/6/4 p /3 /0/0 /1/0 /0 /3/7/5 /"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/:/1/2/1 Example /3/./3/./2/. Let us /nd the singular v alue decomp osition of thematrix A /= h/2 /1 /; /2 i/.I Find the eigen v alues of the matrix A TA /:det /( A TA /; /I /)/= /0 /, /// / / // /4 /; / /2 /; /4/2 /1 /; / /; /2/; /4 /; /2 /4 /; / /// / / // /=/0 /) /8/>/</>/: //1 /=/9 /;;//2 /=/0 /;;//3 /=/0 /:I I Find the n um be r of the nonzero eigen v alues of the matrix A TA /: r /=/1 /:I I I Find the eigen v ector of the matrix A TA /://1 /=/9 /) v/1 /= h/; /2 /= /3 /; /1 /= /3 /2 /= /3 iT/;;//2 /;; /3 /=/0 /) /8/>/</>/: v/2 /= h/; p /5 /= /5 /2 p /5 /= /5 /0 iT/;;v/3 /= h/4 p /5 /= /1/5 /2 p /5 /= /1/5 /5 p /5 /= /1/5 iT/:Since the eigen v alue /0 is m ultiple/, the Gram/-Sc hmidt orthogonalization pro cessis used to /nd the v ector v/3 /. W e compile the orthonormal matrix V /:V /= /2/6/4 /; /2 /= /3 /; p /5 /= /5 /4 p /5 /= /1/5/; /1 /= /3 /2 p /5 /= /5 /2 p /5 /= /1/5/2 /= /3 /0 /5 p /5 /= /1/5 /3/7/5 /:IV F orm the singular v alue matrix/://= h/3 /0 /0 i/:V Calculate the unique column/-v ector of the matrix U applying the form ula/(/9/)/:u/1 /= /1 /3 A v/1 /= /1 /3 h/2 /1 /; /2 ih/; /2 /= /3 /; /1 /= /3 /2 /= /3 iT/= h/; /1 i/:Th us the singular v alue decomp osition of the matrix A isA /= U / V T/= h/; /1 ih/3 /0 /0 i /2/6/4 /; /2 /= /3 /; /1 /= /3 /2 /= /3p /5 /= /5 /2 p /5 /= /5 /0/4 p /5 /= /1/5 /2 p /5 /= /1/5 /5 p /5 /= /1/5 /3/7/5 /:/1/2/2 Example /3/./3/./3/. /Let us /nd the singular v alue decomp osition of thematrixA /= /2/6/4 /2 /2 /2 /2/1/7 /1/0 /1 /1/0 /; /1/7 /1/0 /; /1 /1/0/3 /5 /9 /5 /; /3 /5 /; /9 /5 /3/7/5 /:The giv en /3 / /4 matrix A has three nonzero singular v alues/. Therefore it isenough to /nd nonzero singular v alues of the matrix A using the /3 / /3 matrixAA T/(not the /4 / /4 matrix A TA /)/. SinceAA T/= /2/6/4 /2 /2 /2 /2/1/7 /1/0 /1 /1/0 /; /1/7 /1/0 /; /1 /1/0/3 /5 /9 /5 /; /3 /5 /; /9 /5 /3/7/5 /2/6/6/6/4 /2 /1/7 /1/0 /3 /5/2 /1 /1/0 /9 /5/2 /; /1/7 /1/0 /; /3 /5/2 /; /1 /1/0 /; /9 /5 /3/7/7/7/5 /= /2/6/4 /1/6 /0 /0/0 /2/9 /5 /1/2 /5/0 /1/2 /5 /3/6 /5 /3/7/5 /;;then the c haracteristic equation of AA Tis// / / /// /1/6 /; / /0 /0/0 /2/9 /5 /; / /1/2 /5/0 /1/2 /5 /3/6 /5 /; / // / / /// /=/0or/(/1/6 /; / /) //3/6 /; /1/3 / /+ / /2 //=/0 /;;and the solutions of this equation are //1 /= /1/6 /;; //2 /= /9 and //3 /= /4 /: Since/i /= / /2i and the matrix / is a /3 / /4 matrix/, then on the leading diagonalof the matrix / there are the singular v alues of the matrix A in descendingorder/, and all other elemen ts of the matrix / are zeros/://= /2/6/4 /4 /0 /0 /0/0 /3 /0 /0/0 /0 /2 /0 /3/7/5 /:The matrix U has for column/-v ectors the orthonormed eigen v ectors of thematrix AA T/://1 /=/1 /6 /) u/1 /= h/1 /0 /0 iT/;;//2 /=/9 /) u/2 /= h/0 /3 /5 /4 /5 iT/;;//3 /=/4 /) u/3 /= h/0 /; /4 /5 /3 /5 iT/:/1/2/3 Collecting the v ectors u/1 /;; u/2 and u/3 /;; w e obtain the matrixU /= /2/6/4 /1 /0 /0/0 /3 /5 /; /4 /5/0 /4 /5 /3 /5 /3/7/5 /:According to the relation /(/6/)/, w e shall /nd the /rst three column/-v ectors ofthe matrix V /(the matrix / has three nonzero en tries on its leading diagonal/) using the form ulavi /= /1 /i A Tui /:Hencev/1 /= /2/6/6/6/4 /1 /2/1 /2/1 /2/1 /2 /3/7/7/7/5 /;; v/2 /= /2/6/6/6/4 /1 /2/1 /2/; /1 /2/; /1 /2 /3/7/7/7/5 /;; v/3 /= /2/6/6/6/4 /; /1 /2/1 /2/1 /2/; /1 /2 /3/7/7/7/5 /:T o calculate the v ector v/4 /, w e /nd /rst/, using the Gram/-Sc hmitd orthog/-onalization pro cess/, the v ector bv/4 p erp endicular to the v ectors v/1 /;; v/2 andv/3 /:bv/4 /= e/1 /; /( v T/1 e/1 /) v/1 /; /( v T/2 e/1 /) v/2 /; /( v T/3 e/1 /) v/3 /=/= e/1 /; /1 /2 v/1 /; /1 /2 v/2 /+ /1 /2 v/3 /= h/1 /4 /; /1 /4 /1 /4 /; /1 /4 iT/:Since k bv/4 k/2 /= /1 /2 /;; thenv/4 /=/2 bv/4 /= h/1 /2 /; /1 /2 /1 /2 /; /1 /2 iTandV /= /2/6/6 /6/4 /1 /2 /1 /2 /; /1 /2 /1 /2/1 /2 /1 /2 /1 /2 /; /1 /2/1 /2 /; /1 /2 /1 /2 /1 /2/1 /2 /; /1 /2 /; /1 /2 /; /1 /2 /3/7/7 /7/5 /:Let us c hec k the result/:U / V T/= /2/6/4 /1 /0 /0/0 /3 /5 /; /4 /5/0 /4 /5 /3 /5 /3/7/5 /2/6/4 /4 /0 /0 /0/0 /3 /0 /0/0 /0 /2 /0 /3/7/5 /2/6/6 /6/4 /1 /2 /1 /2 /1 /2 /1 /2/1 /2 /1 /2 /; /1 /2 /; /1 /2/; /1 /2 /1 /2 /1 /2 /; /1 /2/1 /2 /; /1 /2 /1 /2 /; /1 /2 /3/7/7 /7/5 /=/1/2/4 /= /2/6/4 /2 /2 /2 /2/1/7 /1/0 /1 /1/0 /; /1/7 /1/0 /; /1 /1/0/3 /5 /9 /5 /; /3 /5 /; /9 /5 /3/7/5 /= AandU TAV /= /2/6/4 /1 /0 /0/0 /3 /5 /4 /5/0 /; /4 /5 /3 /5 /3/7/5 /2/6/4 /2 /2 /2 /2/1/7 /1/0 /1 /1/0 /; /1/7 /1/0 /; /1 /1/0/3 /5 /9 /5 /; /3 /5 /; /9 /5 /3/7/5 /2/6/6 /6/4 /1 /2 /1 /2 /; /1 /2 /1 /2/1 /2 /1 /2 /1 /2 /; /1 /2/1 /2 /; /1 /2 /1 /2 /1 /2/1 /2 /; /1 /2 /; /1 /2 /; /1 /2 /3/7/7 /7/5 /=/:/= /2/6/4 /4 /0 /0 /0/0 /3 /0 /0/0 /0 /2 /0 /3/7/5 /=/ /:Problem /3/./3/./1/. /Applying the singular v alue decomp osition of the ma/-trix A obtained in example /3/./3/./3/, /nd the bases of the subspace of the column/-v ectors R /( A /) /;; the righ t n ull space N /( A /) /;; the subspace of the ro w/-v ectorsR /( A T/)/, and the left n ull space N /( A T/)o ft h e matrix A /.Problem /3/./3/./2/. /Find the singular v alue decomp osition and the QRfactorization of the matrix A /= /"/3/4 /#/.Problem /3/./3/./3/. /Find the singular v alue decomp osition of the matrixA /= h/; /5 /2 /+/3 p /3 /5 /2 p /3/+ /3 i/./2/./4 Pseudoin v erse Matrix/2/./4/./1 Least/-Squares Metho dLet us consider the solution of a system of linear equationsA x /= b /(/1/)b y the least/-squares metho d in the case where the condition of the Kronec k er/-Cap elli theorem is not satis/ed/, i/.e/./, the system has no solution in an ordinary sense/./1/2/5 Example /4/./1/./1/. Let the system be/2/6/4 a/1/1 a/1/2a/2/1 a/2/2a/3/1 a/3/2 /3/7/5 /"//1//2 /#/= /2/6/4 //1//2//3 /3/7/5 /;;where b /= h//1 //2 //3 iT/= /2R /( A /) and r ank /( A /) /= /2 /: Let p be the orthog/-onal pro jection of the v ector b on to the space R /( A /) /: Since the v ector p /2R /( A /) and ra n k /( A /) /= /2 /;; the system A x /= p has a unique solution/. T ak/-ing in to consideration that R /3/= R /( A /) /N /( A T/) /;; w e get b /; p /2N /( A T/) /,A T/( b /; p /)/= /0 and A T/( b /; A x /)/= /0 orA TA x /= A Tb /: /(/2/)The matrix A TA of the system /(/2/) is regular since r ank /( A /) /= /2 /: Thereforethe system /(/2/) is uniquely solv able on the giv en conditions and x /=/( A TA /) /; /1A Tb /: /(/3/)By minimizing the square of the norm of discrepancy A x /; bk A x /; b k /2/2 /=/( A x /; b /) T/( A x /; b /)/= /( x TA T/; b T/) T/( A x /; b /)/( gr a d k A x /; b k /2/2 /=/0 /) /, w e obtain the same system /(/2/)/, and hence the samesolution x determined b y the form ula /(/3/)/, the le ast/-squar e solution of theequation /(/1/)/.The line of reasoning giv en in example /4/./1/./1 can b e realized also in a moregeneral case/.De/nition /4/./1/./1/. If A /2 R m / n/;; then system /(/2/) is called the system ofnormal e quations of system /(/1/)/.Prop osition /4/./1/./1/. If A /2 R m / n/;; b /= /2R /( A /) and supp ose ra n k /( A /)/= n/;;then the system of normal equations /(/2/) of system /(/1/) is uniquely solv ableand the least/-squares solution x of the system /(/1/) is giv en b y /(/3/)/.Example /4/./1/./2/. /Let us solv eb y the least/-squares metho d the system ofequations/2/6/4 /1 /1/2 /3/2 /1 /3/7/5 /"//1//2 /#/= /2/6/4 /1/1/1 /3/7/5 /:/1/2/6 W e form the system of normal equations A TA x /= A Tb /:/"/1 /2 /2/1 /3 /1 /# /2/6/4 /1 /1/2 /3/2 /1 /3/7/5 x /= /"/1 /2 /2/1 /3 /1 /# /2/6/4 /1/1 /1 /3/7/5 /) /"/9 /9/9 /1/1 /# x /= /"/5/5 /#/:Th us/,/(/9 //1 /+/9 //2 /=/5/9 //1 /+/1 /1 //2 /=/5 /) /( //1 /= /5 /9 //2 /=/0 /:If A /2 R m / n/, b /= /2R /( A /) and r ank /( A /) /< n /, then the system of normalequations /(/2/) has an in/nite n um b er of solutions/, whic h can b e all expressedas x /= xr /+ xn /;;where xr /2R /( A T/)a n d xn /2N /( A /) /: F rom among the solutions x w e will /ndthe one ha ving the least norm/, the so/-called optim um solution x /+/: F rom theorthogonalit yo f t h e v ectors xr and xn it follo ws thatk x k /2/2 /= k xr k /2/2 /+ k xn k /2/2 /:Since from xn /2N /( A /) it follo ws A xn /=/0 /;; thenA x /= p /, A /( xr /+ xn /)/= p /, A xr /+ A xn /= p /) A xr /= pand xr /2R /( A T/) is the optim um solution x /+of the equation A x /= p /. Th us/,x /+/= x /: /2/2/./4/./2 Pseudoin v erse Matrix and Optim um SolutionNext w e will consider the algorithm for /ndig the optim um solution/.Example /4/./2/./1/. Let b /= h//1 //2 //3 iTand//= /2/6/4 //1 /0 /0 /0/0 //2 /0 /0/0 /0 /0 /0 /3/7/5 /;;/1/2/7 where //1 /6/=/0 a n d //2 /6/=/0 /: W e will /nd the optim um solution of the system/ x /= b /:The orthogonal pro jection of the v ector b on the space R /(//) is p /= h//1 //2 /0 iT/,and b /; p /= h/0 /0 //3 iT/: T o /nd the solution x /, one m ust solv e the system/ x /= p /;;i/.e/./,/2/6/4 //1 /0 /0 /0/0 //2 /0 /0/0 /0 /0 /0 /3/7/5 /2/6/6/6/4 //1 //2 //3 //4 /3/7/7/7/5 /= /2/6/4 //1//2/0 /3/7/5or/8/>/</>/: //1 //1 /+/0 //2 /+/0 //3 /+/0 //4 /= //1/0 //1 /+ //2 //2 /+/0 //3 /+/0 //4 /= //1/0 //1 /+/0 //2 /+/0 //3 /+/0 //4 /=/0 /) /8/>/> /> /</>/> /> /: //1 /= //1 /=//1 //2 /= //2 /=//2 //3 /= / //4 /= / /;;where / /;; / /2 R are arbitrary /. T aking / /= / /=/0 /;; w e obtain the solution withthe least /2/-normx /+/= h//1 /=//1 //2 /=//2 /0 /0 iT/:W e state that x /+can b e expressed also b yx /+/= /2/6/6 /6/4 //1 /=//1//2 /=//2/0/0 /3/7/7 /7/5 /= /2/6/6 /6/4 /1 /=//1 /0 /0/0 /1 /=//2 /0/0 /0 /0/0 /0 /0 /3/7/7 /7/5 /2/6/4 //1//2//3 /3/7/5 /:The optim um solution x /+of the giv en example can be obtained from thev ector b b y m ultiplying it on the left b y the matrix/ /+/= /2/6/6 /6/4 /1 /=//1 /0 /0/0 /1 /=//2 /0/0 /0 /0/0 /0 /0 /3/7/7 /7/5 /:The matrix / /+is obtained from the matrix / b y transp osing and afterw ardsreplacing the nonzero en tries b y their recipro cals/. Hence x /+/= A /+b /:/1/2/8 Let us generalize the result obtained in example /4/./2/./1/.Prop osition /4/./2/./1/. If//= diag /( //1 /;;/:/:/: /;;/p /) /2 R m / n/( p /= min f m/;; n g /) /(/1/)and//1 / //2 / /:/:/: / /r />/r /+/1 /= /:/:/: /= /p /;; /(/2/)then the optim um solution x /+of the system/ x /= bis giv en b yx /+/=/ /+b /;;where/ /+/= diag /(/1 /=//1 /;;/:/:/: /;; /1 /=/r /;; /0 /;;/:/:/: /;; /0/) /2 R n / m/: /(/3/)De/nition /4/./2/./1/. LetA /= U / V Tb e the singular v alue decomp osition of the matrix A /2 R m / n/. The pseudoin/-verse matrix of the matrix A is a matrixA /+/= V / /+U T/;;where / and / /+are giv en b y relations /(/1/-/3/)/.Problem /4/./2/./1/. Let A /2 R n / nand det /( A /) /6/=/0 /: Sho w that A /+/= A /; /1/:Problem /4/./2/./2/. Let us /nd the pseudoin v erse matrix of the matrixA /= h/2 /1 /; /2 igiv en in example /3/./3/./2/. W e found the singular v aluedecomp osition of the matrix A in this exampleA /= U / V T/= h/; /1 ih/3 /0 /0 i /2/6/4 /; /2 /= /3 /; /1 /= /3 /2 /= /3p /5 /= /5 /2 p /5 /= /5 /0/4 p /5 /= /1/5 /2 p /5 /= /1/5 /5 p /5 /= /1/5 /3/7/5 /:Using de/nition /4/./2/./1/,A /+/= V / /+U T/;;i/.e/./,A /+/= /2/6/4 /; /2 /= /3 p /5 /= /5 /4 p /5 /= /1/5/; /1 /= /3 /2 p /5 /= /5 /2 p /5 /= /1/5/2 /= /3 /0 /5 p /5 /= /1/5 /3/7/5 /2/6/4 /1 /= /3/0/0 /3/7/5 h/; /1 i/= /2/6/4 /2 /= /9/; /1 /= /9/; /2 /= /9 /3/7/5 /:/1/2/9 Prop osition /4/./2/./2/. If A /2 R m / n/;; then the optim um solution x /+of thesystem A x /= p /(in the sense of least/-squares/) is giv en b yx /+/= A /+b /:Pr o of/. When a v ector is m ultiplied b y the orthogonal matrix U T/, its/2/-norm is conserv ed/. Therefore/,k A x /; b k/2 /= / / / U / V Tx /; b / / / /2 /= / / / / V Tx /; U Tb / / / /2 /:Let substitute y /= V Tx /: Henceminx /2 R n k A x /; b k/2 /= miny /2 R n / / / / y /; U Tb / / / /2 /:Prop osition /4/./2/./1 implies that the minimizing v ector for the expression / / / / y /; U Tb / / / /2is the v ectory /+/=/ /+U Tband the v ectorx /+/= V y /+/= V / /+U Tb /= A /+bminimizes the expression k A x /; b k/2 /: /2Example /4/./2/./3/. Let us /nd the optim um solution of the system/2 //1 /+ //2 /; /2 //3 /=/9 /:In example /4/./2/./2/, w e found the pseudoin v erse matrixA /+/= /2/6/4 /2 /= /9/; /1 /= /9/; /2 /= /9 /3/7/5of the matrix of the system A /= h/2 /1 /; /2 i/:In virtue of prop osition /4/./2/./2/, w e get the optim um solutionx /+/= A /+b /= /2/6/4 /2 /= /9/; /1 /= /9/; /2 /= /9 /3/7/5 h/9 i/= /2/6/4 /2/; /1/; /2 /3/7/5 /:/1/3/0 Example /4/./2/./4/. Let us /nd the optim um solution of the system/2/6/4 /1 /1/0 /1/1 /0 /3/7/5 x /= /2/6/4 /1/2 /3 /3/7/5 /:In example /3/./3/./1/, w e found the singular v alue decomp osition of the systemmatrix AA /= /2/6/4 p /6 /= /3 /0 p /3 /= /3p /6 /= /6 p /2 /= /2 /; p /3 /= /3p /6 /= /6 /; p /2 /= /2 /; p /3 /= /3 /3/7/5 /2/6/4 p /3 /0/0 /1/0 /0 /3/7/5 /"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/:Using de/nition /4/./2/./1/, w e will /nd the pseudoin v erse matrixA /+/= /"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/"/1 /= p /3 /0 /0/0 /1 /0 /# /2/6/4 p /6 /= /3 p /6 /= /6 p /6 /= /6/0 p /2 /= /2 /; p /2 /= /2p /3 /= /3 /; p /3 /= /3 /; p /3 /= /3 /3/7/5 /=/= /"/1 /= /3 /2 /= /3 /; /1 /= /3/1 /= /3 /; /1 /= /3 /2 /= /3 /#/:The optim um solution of the system will b ex /+/= A /+b /= /"/1 /= /3 /2 /= /3 /; /1 /= /3/1 /= /3 /; /1 /= /3 /2 /= /3 /# /2/6/4 /1/2/3 /3/7/5 /= /"/2 /= /3/5 /= /3 /#/:Problem /4/./2/./2/. /Find the pseudoin v erse of the matrix A /= /[/0/] andexplain the result/. A nswer/: A /+/= /[/0/] /:Problem /4/./2/./3/. /Find the pseudoin v erse of the matrix Aa /) A /= /"/3/4 /#/;; b /) A /= /2/6/4 /1 /1/2 /3/2 /1 /3/7/5 /:Problem /4/./2/./4/. /What is the pseudoin v erse matrix of the matrix A withorthogonal columns/? A nswer/: A /+/= A T/:/1/3/1 Problem /4/./2/./5/. /Find the optim um solution of the system/2/6/4 /1 /1/2 /3/2 /1 /3/7/5 /"//1//2 /#/= /2/6/4 /1/1 /1 /3/7/5 /:Prop osition /4/./2/./3 /( Conditions of Mo or e/-Penr ose/. /) If A /2 R m / n/;; thenthe conditionsAX A /= A/;; XA X /= X/;; /( AX /) T/= AX /;; /( XA /) T/= XAare satis/ed only b y one matrix X /2 R n / m/, and this is A /+/:Problem /4/./2/./6/. /A matrix A is called a pr oje ctionmatrix ifA /2/= A /^ A T/= A/:Chec k the Mo ore/-P enrose conditions for the pro jectionmatrix/. Do es A /+/=A /?/2/./5 Jordan F orm of a Matrix/2/./5/./1 Matrix DiagonalizationIn prop osition /1/./2/./6/./8 on the Jordan decomp osition it is stated that ifA /2 C n / n/;; then there exists suc h a regular X /2 C n / nthatX /; /1AX /= J /= diag /( J/1 /;; /:/:/: /;; Jt /) /;; /(/1/)where m/1 /+ /:/:/: /+ mt /= n andJi /= /2/6/6/6/6/6 /6 /6/6/4 /i /1 /0 /
/
/
/0/0 /i /1 /././. /./././././. /././. /././. /././. /./././././. /././. /././. /./././1/0 /
/
/
/
/
/
/0 /i /3/7/7/7/7/7 /7 /7/7/5 /2 C mi / mi/1/3/2 is a Jor dan blo ck or Jor dan b ox /, and the matrix J is called a Jor dan c anonic alform or Jordan normal form of the matrix A /. The n um be r of Jordan blo c ksin decomp osition /(/1/) equals the n um be r of the linearly indep enden t eigen/-v ectors of the matrix A /. Namely /, to eac h linearly indep enden t eigen v ectorcorresp onds one blo c k/. Hence if the matrix A has a basis of eigen v ectors/,then all the Jordan blo c ks are /1 / /1 blo c ks/, and the Jordan normal formcoincides with the diagonal form of the matrix giv en in prop osition /1/./2/./5/./8S /; /1AS /=/ /;; where //= diag /( //1 /;;/:/:/: /;;/n /) and the matrix S has for columnsthe linearly indep enden t eigen v ectors of the matrix A corresp onding to theseeigen v alues/.Example /5/./1/./1/. /Let us /nd the Jordan form of the matrixA /= /2/6/4 /3 /; /1 /2/1 /; /1 /1/; /1 /1 /0 /3/7/5 /:W e shall /nd the eigen v alues of the matrix A /:det/( A /; /I /)/= /0 /, /( / /2/; /1/)/( / /; /2/) /) /8/>/</>/: //1 /=/1 /;;//2 /= /; /1 /;;//3 /=/2 /:No w w e shall /nd the eigen v ectors corresp onding to these eigen v alues/://1 /=/1 /! /2/6/6 /6/4 /3 /; /1 /; /1 /2 /././. /0/1 /; /1 /; /1 /1 /././. /0/; /1 /1 /0 /; /1 /././. /0 /3/7/7 /7/5 I /$ II// /2/6/6 /6/4 /1 /; /2 /1 /./. /. /0/2 /; /1 /2 /./. /. /0/; /1 /1 /; /1 /././. /0 /3/7/7 /7/5 II /; /2 /
I/III /+ I /2/6/6 /6/4 /1 /; /2 /1 /./. /. /0/0 /3 /0 /./. /. /0/0 /; /1 /0 /././. /0 /3/7/7 /7/5 /) x/1 /= /2/6/4 /1/0/; /1 /3/7/5 /;;//2 /= /; /1 /! x/2 /= /2/6/4 /; /1/; /2/1 /3/7/5 /;; //3 /=/2 /! x/3 /= /2/6/4 /5/1/; /2 /3/7/5 /:W e compile the matrix of eigen v ectors of the matrix AS /= hx/1 x/2 x/3 i/= /2/6/4 /1 /; /1 /5/0 /; /2 /1/; /1 /1 /; /2 /3/7/5/1/3/3 and /nd the in v erse matrixS /; /1/= /2/6/4 /; /1 /2 /; /1 /2 /; /3 /2/1 /6 /; /1 /2 /1 /6/1 /3 /0 /1 /3 /3/7/5 /:As the result/, w e obtainS /; /1AS /= /2/6/4 /; /1 /2 /; /1 /2 /; /3 /2/1 /6 /; /1 /2 /1 /6/1 /3 /0 /1 /3 /3/7/5 /2/6/4 /3 /; /1 /2/1 /; /1 /1/; /1 /1 /0 /3/7/5 /2/6/4 /1 /; /1 /5/0 /; /2 /1/; /1 /1 /; /2 /3/7/5 /=/= /2/6/4 /1 /0 /0/0 /; /1 /0/0 /0 /2 /3/7/5 /=/ /:Prop osition /5/./1/./1/. An y Hermitian /(symmetric/) matrix A /2 C n / n/( A /2R n / n/) can b e diagonalized using a unitary matrix U /2 C n / n/(an orthogonalmatrix Q /2 R n / n/) /;; i/.e/./, there exists suc h U /2 C n / n/( Q /2 R n / n/) /;; thatU HAU /=/ /( Q TAQ /=/ /) /: /(/2/)Pr o of/. The Sc h ur factorisation /(prop osition /1/./2/./6/./5/) implies that the Her/-mitian matrix A /2 C n / ncan b e giv en in the formU HAU /= T/;; /(/3/)where U /2 C n / nis a unitary matrix and T /2 C n / nis an upp er triangularmatrix/. Finding the transp ose conjugate matrices of b oth sides of /(/3/)/, w egetU HA HU /= T H/:In virtue of the Hermitian matrix de/nition A H/= A /, w e /nd thatU HAU /= T H/: /(/4/)F rom /(/3/) and /(/4/) it follo ws that T /= D/: The diagonal elemen ts of the diagonalmatrix D similar to the matrix A are the eigen v alues of the matrix A /. Theassertion ab out the symmetric matrix A /2 R n / nis a sp ecial case of thecomplex v ersion/. /2/1/3/4 Problem /5/./1/./1/. /LetA /= /2/6/6 /6/4 /0 /1 /2 /0/1 /; /1 /1 /; /1/2 /1 /1 /; /2/0 /; /1 /; /2 /0 /3/7/7 /7/5 /:Find suc h an orthogonal matrix Q /2 R /4 / /4/;; that Q TAQ /= / /;; where / is adiagonal matrix/.Problem /5/./1/./2/. /LetA /= /2/6/4 /1 i /1/+ i/; i /; /1 /1/1 /; i /1 /0 /3/7/5 /:Find suc h a unitary matrix U /2 C n / n/;; that U HAU /= / /;; where / is adiagonal matrix/.Not ev ery square matrix can be put in form /(/2/)/. Prop osition /1/./2/./6/./6implies that only a normal matrix A /( A HA /= AA H/) can be expressed inform /(/2/)/. In the general case of the diagonalization of a matrix one m ustcon/ne himself to the Jordan normal form /(/1/)/./2/./5/./2 Analysis of Jordan F orm of a MatrixIt is not su/cien t to /nd the eigen v alues of the matrix to obtain theJordan form of this matrix/.Example /5/./2/./1/. Let us /nd the Jordan matrices J of the matricesT /= /"/1 /2/0 /1 /#/;; A /= /"/2 /; /1/1 /0 /#/;; B /= /"/1 /0/1 /1 /#/;; I /= /"/1 /0/0 /1 /#/:It is easy to /nd out that the sp ectra of T/;; A/;; B and I are the same/,/ /( T /) /= / /( A /) /= / /( B /) /= / /( I /) /= f /1/;; /1 g /: Let us /nd the eigen v ectors corre/-sp onding to / /=/1 /:T /! /2/4 /0 /2 /././. /0/0 /0 /././. /0 /3/5/! x/1 /= /"/1/0 /#/;;/1/3/5 A /! /2/4 /1 /; /1 /./. /. /0/1 /; /1 /././. /0 /3/5/! x/1 /= /"/1/1 /#/;;B /! /2/4 /0 /0 /././. /0/1 /0 /././. /0 /3/5/! x/1 /= /"/0/1 /#/;;I /! /2/4 /0 /0 /././. /0/0 /0 /./. /. /0 /3/5/! x/1 /= /"/1/0 /#/^ x/2 /= /"/0/1 /#/:W e see that the matrices T/;; A and B ha v e only one indep enden t eigen v ectorand only one Jordan blo c k corresp onding to the eigen v alue / /=/1 /, and th usthe matrices T/;; A and B ha v e the same Jordan matrixJ /= /"/1 /1/0 /1 /#/:The matrix I has t w o linearly indep enden t eigen v ectors and/, consequen tly /,t w o Jordan blo c ks/, and the corresp onding Jordan matrix coincides with thematrix I/:Problem /5/./2/./1/. V erify that the matrix/2/6/6/6 /6/4 /1 /1 /
/
/
/1/0 /1 /
/
/
/1/././. /././. /././. /./././0 /0 /
/
/
/1 /3/7/7/7 /7/5 /2 R n / ncorresp onds the one/-blo c k Jordan matrix/2/6/6/6/6 /6 /6 /6 /6/4 /1 /1 /0 /
/
/
/0/0 /1 /1 /./././0/0 /0 /1 /././. /./. /./././. /././. /././. /././. /./././0 /0 /0 /
/
/
/1 /3/7/7/7/7 /7 /7 /7 /7/5 /2 R n / n/:Example /5/./2/./2/. Let us consider the Jordan matrixJ /= /2/6/6 /6 /6 /6 /6/4 /3 /1 /0 /0 /0/0 /3 /0 /0 /0/0 /0 /0 /1 /0/0 /0 /0 /0 /0/0 /0 /0 /0 /0 /3/7/7 /7 /7 /7 /7/5 /= /2/6/4 J/1J/2J/3 /3/7/5 /: /(/5/)/1/3/6 Let us /nd the eigen v ectors corresp onding to the eigen v alue / /=/3 of m ulti/-plicit y/2 /:/2/6/6 /6/6/6 /6 /6 /6 /6/6/4 /0 /1 /0 /0 /0 /././. /0/0 /0 /0 /0 /0 /./. /. /0/0 /0 /; /3 /1 /0 /./. /. /0/0 /0 /0 /; /3 /0 /./. /. /0/0 /0 /0 /0 /; /3 /./. /. /0 /3/7/7 /7/7/7 /7 /7 /7 /7/7/5 /) x /= /2/6/6/6 /6 /6 /6/4 p/0/0/0/0 /3/7/7/7 /7 /7 /7/5 /:Therefore/, one linearly indep enden t eigen v ector e/1 and one Jordan blo c kcorresp onds to the eigen v alue / /=/3 /:/"/3 /1/0 /3 /#/:Let us /nd the eigen v ectors corresp onding to the eigen v alue / /=/0 of m ulti/-plicit y/3 /:/2/6/6/6 /6 /6 /6/6/6 /6 /6/4 /3 /1 /0 /0 /0 /./. /. /0/0 /3 /0 /0 /0 /./. /. /0/0 /0 /0 /1 /0 /././. /0/0 /0 /0 /0 /0 /././. /0/0 /0 /0 /0 /0 /././. /0 /3/7/7/7 /7 /7 /7/7/7 /7 /7/5 /) x /= /2/6/6 /6 /6/6/6/4 /0/0q/0r /3/7/7 /7 /7/7/7/5 /:Hence t w o linearly indep enden t eigen v ectors e/3 and e/5 and t w o Jordan blo c kscorresp ond to the eigen v alue / /=/0/"/0 /1/0 /0 /#and/[/0/] /:The question arises/, what conditions m ust the the /5 / /5 matrix A satisfy toha v e for the corresp onding Jordan matrix the J giv en b y /(/5/)/? Ho w do w e/nd the regular matrix X suc h thatX /; /1AX /= J /? /(/6/)/1/3/7 The /rst condition is / /( A /)/= / /( J /) /;; but it is not su/cien t/. The eigen v aluesof the matrix A m ust be also considered/. W e express the relation /(/6/) in theform AX /= XJ orA hx/1 /
/
/
x/5 i/= hx/1 /
/
/
x/5 i /2/6/6 /6 /6/6/6/4 /3 /1/3/0 /1/0/0 /3/7/7 /7 /7/7/7/5 /:Ha ving m ultiplied the matrices/, w e get the form ulasA x/1 /=/3 x/1 /;; A x/2 /=/3 x/2 /+ x/1 /(/7/)andA x/3 /=/0 x/3 /;; A x/4 /=/0 x/4 /+ x/3 /;; A x/5 /=/0 x/5 /:/(/8/)F rom the form ulas /(/7/) and /(/8/) it follo ws that similarly to the matrix J thematrix A m ust ha v e three eigen v ectors x/1 /;; x/3 and x/5 /: In addition/, the matrixA m ust ha v e t w o gener alize d eigen v ectors or t w o /rst or der / ag ve ctors x/2and x/4 /: It is said that the v ector x/2 b elongs to the c hain that b egins with thev ector x/1 and is de/ned b y the form ula /(/7/)/. This c hain determines the Jordanblo c k J/1 /: The t w o /rst form ula of /(/8/) de/ne the second c hain consisting ofthe v ectors x/3 and x/4 /, and this c hain/, in its turn/, de/nes the Jordan blo c k J/2 /:The last of the form ulas /(/8/) de/nes the third c hain consisting of the v ectorx/5 /, and this c hain/, in its turn/, de/nes the Jordan blo c k J/3 /:Prop osition /5/./2/./1/. The determination of the Jordan form of the ma/-trix A /2 C n / nreduces to the /nding of c hains/. Ev ery c hain starts on theeigen v ector of the matrix A and for ev ery v alue of the index i /=/1 /: nA xi /= /i xi /_ A xi /= /i xi /+ xi /; /1 /: /(/9/)The v ectors xi are the column v ectors of the matrix X /, and ev ery c haindetermines one Jordan blo c k/./2/./5/./3 Algorithm of Filip o v/1/3/8 If n /= /1/, then the Jordan blo c k coincides with the giv en matrix andform ula /(/9/) is true/. Let us supp ose that the Jordan form of the matrix A isfound b y applying the Jordan blo c k construction form ula /(/9/) if the order ofthe matrix A is smaller than n/: w e will use mathematical induction/.I step/. Assume that A is singular/, dim R /( A /) /= r /< n/: Considering thecorresp onding r / r matrix w e /nd that in this case the construction based onform ulas /(/9/) is realizable/. Namely /, in the space R /( A /) there are r indep enden tv ectors wi suc h that the follo wing relations holdA wi /= /i wi /_ A wi /= /i wi /+ wi /; /1 /: /(/1/0/)II step/. Let us supp ose that dim R /( A /) /\N /( A /) /= p/: Ev ery v ector ofthe n ull space N /( A /) is an eigen v ector of the matrix A corresp onding to theeigen v alue of the matrix A / /=/0 /: Therefore/, there m ust be p c hains on theI step whic h be g i n with the eigen v ectors corresp onding to the eigen v alue /0/.W e are in terested in the last v ector of eac h suc h c hain/. Since the v ectors wib elonging to the subspace R /( A /) /\N /( A /)m ust also b elong to the space R /( A /) /;;then they ha v e to be the linear com binations of the column v ectors of thematrix Awi /= A yiwith some yi /. Therefore/, the v ector yi follo ws the v ector wi in the c haincorresp onding to the eigen v alue / /=/0 /.III step/. Since dim N /( A /)/= n /; p/;; there m ust b e n /; r /; p more linearlyindep enden t v ectors zi of the space N /( A /) in the orthogonal complemen t ofthe subspace R /( A /) /\N /( A /)/.Prop osition /5/./3/./1/. The algorithm of Filip o v de/nes r v ectors wi /;; pv ectors yi and n /; r /; p v ectors zi /;; whic h determine the Jordan c hains/. Thesev ectors are linearly indep enden t/, they can be c hosen for the column/-v ectorsof the matrix X /, and J /= X /; /1AX /:Pr o of/. See Strang /(/1/9/8/8/, p/. /4/5/7/)/. /2Example /5/./3/./1/. Let us /nd the Jordan normal form of the matrixA /= /2/6/4 /0 /1 /2/0 /0 /0/0 /0 /0 /3/7/5using the algorithm of Filip o v/./1/3/9 I step/. F rom the form of the matrix / /( A /) /= f /0/;; /0/;; /0 g and R /( A /) /=span f e/1 g /: Hence r /= /1 and there is a v ector w/1 /= e/1 from this subspaceR /( A /) satisfying the condition /(/1/0/)/.II step/. Let us /nd the basis of the n ull space N /( A /)o ft h e matrix A /:/2/6/6 /6/4 /0 /1 /2 /././. /0/0 /0 /0 /././. /0/0 /0 /0 /././. /0 /3/7/7 /7/5 /) n/1 /= /2/6/4 /1/0/0 /3/7/5 /^ n/2 /= /2/6/4 /0/2/; /1 /3/7/5 /:The v ector n/1 b elongs to the subspace R /( A /) /\N /( A /) and p /= dim R /( A /) /\N /( A /)/=/1 /: W e solv e the system/2/6/6 /6/4 /0 /1 /2 /././. /1/0 /0 /0 /././. /0/0 /0 /0 /././. /0 /3/7/7 /7/5 /) y/1 /= /2/6/4 /0/1 /0 /3/7/5 /:III step/. W e tak e for the v ector z/1 the v ector n/2 and form the matrixX /:X /= hw/1 y/1 z/1 i/= /2/6/4 /1 /0 /0/0 /1 /2/0 /0 /; /1 /3/7/5 /:No w w e /nd the in v erse matrix/2/6/6 /6/4 /1 /0 /0 /./. /. /1 /0 /0/0 /1 /2 /././. /0 /1 /0/0 /0 /; /1 /././. /0 /0 /1 /3/7/7 /7/5 / /2/6/6 /6/4 /1 /0 /0 /./. /. /1 /0 /0/0 /1 /0 /././. /0 /1 /2/0 /0 /1 /././. /0 /0 /; /1 /3/7/7 /7/5 /)X /; /1/= /2/6/4 /1 /0 /0/0 /1 /2/0 /0 /; /1 /3/7/5and the Jordan matrixX /; /1AX /= /2/6/4 /1 /0 /0/0 /1 /2/0 /0 /; /1 /3/7/5 /2/6/4 /0 /1 /2/0 /0 /0/0 /0 /0 /3/7/5 /2/6/4 /1 /0 /0/0 /1 /2/0 /0 /; /1 /3/7/5 /= /2/6/4 /0 /1 /0/0 /0 /0/0 /0 /0 /3/7/5 /:/1/4/0 The soft w are pac k age /\Maple/" giv es for the Jordan decomp osition/:/2/6/4 /0 /1 /2/0 /0 /0/0 /0 /0 /3/7/5 /= /2/6/4 /2 /1 /; /1 /2/0 /0 /1/0 /1 /; /1 /2 /3/7/5 /2/6/4 /0 /1 /0/0 /0 /0/0 /0 /0 /3/7/5 /2/6/4 /1 /2 /0 /; /1 /2/0 /1 /2 /1/0 /1 /0 /3/7/5 /:Since the matrix X in the Jordan decomp osition of the matrix A is notuniquely de/ned/, then for man y problems it is of in terest to c ho ose the matrixX so that the conditional n um be r k /( X /) is the least/. Suc h a problem arosein example /1/./2/./9/./4/.Problem /5/./3/./1/. Find the Jordan decomp osition of the matrixA /= /2/6/6 /6/4 /3 /1 /0 /0/; /4 /; /1 /0 /0/7 /1 /2 /1/; /1/7 /; /6 /; /1 /0 /3/7/7 /7/5 /:Problem /5/./3/./2/. /Find the Jordan decomp osition of the matrixA /= /2/6/6 /6/4 /2 /1 /2 /0/; /2 /2 /1 /2/; /2 /; /1 /; /1 /1/3 /1 /2 /; /1 /3/7/7 /7/5 /:Problem /5/./3/./3/. /Find the Jordan decomp osition of the matrixA /= /2/6/4 /2 /0 /0/1 /1 /; /1/; /1 /1 /3 /3/7/5 /:Problem /5/./3/./4/. Let the Jordan decomp osition of the matrix A /2 R n / nbe A /= MJ M /; /1/: Sho w that A /2/= A /) J /2/= J /:/2/./6 Strict Metho ds of Solving Linear Algebraic Sys/-tems of Equations/2/./6/./1 LD M TDecomp osition and LD L TDecomp osition of a Matrix/1/4/1 Next w e will consider the sp ecial cases of LU factorizations of squarematrices/.Prop osition /6/./1/./1/. If all the principal minors of the matrix A /2 R n / nare di/eren t from zero/, then there exist lo w er triangular matrices L and Mwith the unit leading diagonal and a diagonal matrix D /= diag /( d/1 /;;/:/:/: /;;dn /)thatA /= LD M T/;; /(/1/)and the decomp osition /(/1/) is unique/.Pr o of/. Since all the principal minors of the matrix A /2 R n / nare nonzero/,then the prop osition /1/./2/./2 implies that there exists a unique LU factorizationof the matrix AA /= LU/: /(/2/)Let D /= diag /( d/1 /;;/:/:/: /;;dn /) /;; where di /= uii /(i /=/1/: n/)/. F rom the regularit y ofthe matrix A it follo ws that the matrix D is regular/. Therefore/, /9 D /; /1andM T/= D /; /1U is an upp er triangular matrix/. HenceA /= LU /= LD /( D /; /1U /)/= LD M T/:The uniqueness of the decomp osition /(/1/) follo ws from the uniqueness of thefactorization /(/2/)/. /2De/nition /6/./1/./1/. The decomp osition /(/1/) is called the LD M Tde c omp o/-sition of the regular matrix A /2 R n / n/.Example /6/./1/./1/. /Let us /nd the LD M Tdecomp osition of the matrixA /= /2/6/6 /6/4 /1 /2 /0 /1/; /1 /; /1 /; /3 /0/; /1 /; /3 /2 /2/2 /4 /0 /1 /3/7/7 /7/5 /:W e state that if the principal minors of the matrix A are di/eren t from zero/,then/, b y transforming the matrix A to the triangular form b y the Gausstransformation/, w e /nd sim ultaneously b oth the matrix L and the matrixU /. Namely /,t h ee n try lij /( i /> j /) of the lo w er triangular matrix L equals thefactor b yw h i c ht h e j /-th ro wm ust b e m ultiplied when it is substracted from/1/4/2 the i /; th ro w to delete the en try in the i /; th ro w/. W e /nd/2/6/6/6/4 /1 /2 /0 /1/; /1 /; /1 /; /3 /0/; /1 /; /3 /2 /2/2 /4 /0 /1 /3/7/7/7/5 l/2/1 /= /; /1l/3/1 /= /; /1l/4/1 /=/2/; /! /2/6/6/6/4 /1 /2 /0 /1/0 /1 /; /3 /1/0 /; /1 /2 /3/0 /0 /0 /; /1 /3/7/7/7/5 l/3/2 /= /; /1l/4/2 /=/0/; /!/2/6/6 /6/4 /1 /2 /0 /1/0 /1 /; /3 /1/0 /0 /; /1 /4/0 /0 /0 /; /1 /3/7/7 /7/5 l/4/3 /=/0/; /! /2/6/6 /6/4 /1 /2 /0 /1/0 /1 /; /3 /1/0 /0 /; /1 /4/0 /0 /0 /; /1 /3/7/7 /7/5 /= UandL /= /2/6/6 /6/4 /1 /0 /0 /0/; /1 /1 /0 /0/; /1 /; /1 /1 /0/2 /0 /0 /1 /3/7/7 /7/5 /:Let us c hec k/:LU /= /2/6/6/6/4 /1 /0 /0 /0/; /1 /1 /0 /0/; /1 /; /1 /1 /0/2 /0 /0 /1 /3/7/7/7/5 /2/6/6/6/4 /1 /2 /0 /1/0 /1 /; /3 /1/0 /0 /; /1 /4/0 /0 /0 /; /1 /3/7/7/7/5 /= /2/6/6/6/4 /1 /2 /0 /1/; /1 /; /1 /; /3 /0/; /1 /; /3 /2 /2/2 /4 /0 /1 /3/7/7/7/5 /= A/:Prop osition /6/./1/./2/. If the regular matrix A /2 R n / nis symmetric andthe LD M Tdecomp osition of it has the form /(/1/)/, then L /= M/;; i/.e/./,A /= LD L T/: /(/3/)Pr o of/. F rom decomp osition /(/1/) it follo ws thatAM /; T/= LD /:Multiplying b oth sides of the last equalit y on the left b y matrix M /; /1/;; w e getM /; /1AM /; T/= M /; /1LD /: /(/4/)The matrix M /; /1AM /; Tis symmetric since/( M /; /1AM /; T/) T/= M /; /1A TM /; T/= M /; /1AM /; T/:/1/4/3 The matrix M /; /1AM /; Tis a lo w er triangular matrix since bo t h M /; /1andAM /; T/= LD are lo w er triangular matrices/. In virtue of relation /(/4/)/, thematrix M /; /1LD is also symmetric and lo w er triangular/. Therefore/, the matrixM /; /1LD is diagonal/. Since the matrix D is regular/, then also the matrixM /; /1L is diagonal/. In addition/, the matrix M /; /1L is a lo w er triangular matrixwith the unit diagonal/. Hence M /; /1L /= I or L /= M/: /2Problem /6/./1/./1/. /Find the LU factorization/, LD M Tdecomp osition andLD L Tdecomp osition of the matrixA /= /2/6/6 /6/4 /1 /; /1 /2 /0/; /1 /2 /; /3 /1/2 /; /3 /1 /3/0 /1 /3 /; /4 /3/7/7 /7/5 /:/2/./6/./2 P ositiv e De/nite SystemsDe/nition /6/./2/./1/. The matrix A /2 R n / nis a p ositive de/nite matrix ifx TA x /> /0for all nonzero v ector x /2 R n/.Example /6/./2/./1/. The matrixA /= /"/2 /1/1 /1 /#is a p ositiv e de/nite one since for /8 x /= h//1 //2 iT/2 R /2x TA x /= h//1 //2 i /"/2 /1/1 /1 /#/"//1//2 /#/= h//1 //2 i /"/2 //1 /+ //2//1 /+ //2 /#/=/=/2 / /2/1 /+/2 //1 //2 /+ / /2/2 /= / /2/1 /+/( //1 /+ //2 /) /2/> /0 /:Problem /6/./2/./1/. /Sho w that the matrixA /= /2/6/4 /3 /2 /1/2 /2 /1/1 /1 /1 /3/7/5/1/4/4 is p ositiv e de/nite/.Prop osition /6/./2/./1/. If A /2 R n / nis a p ositiv e de/nite matrix and thecolumn v ectors of the matrix X /2 R n / kare linearly indep enden t/, then thematrixB /= X TAX /2 R k / kis also p ositiv e de/ned/.Pr o of/. If for the v ector z /2 R kthe relation/0 / z TB zholds/, then/0 / z TB z /= z TX TAX z /=/( X z /) TA /( X z /)and from the p ositiv e de/niteness of the matrix A it follo ws that X z /= /0 /:Since the column v ectors of the matrix X are linearly indep enden t/, thenfrom X z /= /0 it follo ws that z /= /0 /: Hence from conditions z /2 R kand z /6/=/0it follo ws that z TB z /> /0 /;; i/.e/./, the matrix B is p ositiv e de/nite/. /2Corollary /6/./2/./1/. If the matrix A /2 R n / nis p ositiv e de/nite/, then allthe submatrices of the matrix A obtained b y deleting the ro ws and columnsof the matrix A with the same n um b ers are p ositiv e de/nite and all theelemen ts on the leading diagonal of the matrix are p ositiv e/.Pr o of/. If v /2 R k/( k / n /) is a v ector with natural n um be r co ordinatessatisfying the condition/1 / //1 /< /: /: /:/</k / n/;;thenX /= In /(/: /;; v /) /2 R n / kis a matrix obtaines from the unit matrix In b y taking the column/-v ectorswith indices //1 /;;/:/:/: /;;/k /: Hence the column/-v ectors of the matrix X are linearlyindep enden t/, and prop osition /6/./2/./1 implies that the matrix X TAX is p ositiv ede/nite/. The matrix X TAX is a submatrix of the matrix A obtained fromthe ro ws and columns with n um b ers //1 /;;/:/:/: /;;/k of the matrix A /. Therefore/, allthe submatrices of the matrix A obtained b y deleting the ro ws and columnsof the matrix A with the same n um be r s are p ositiv e de/nite/. T aking k /=/1 /;;w e get the second part of the statemen t/. /2/1/4/5 Corollary /6/./2/./2/. If A /2 R n / nis p ositiv e de/nite/, then the matrix A hasa decomp osition A /= LD M Tand all the leading diagonal elemen ts of thematrix D are p ositiv e/.Pr o of/. On the ground of corollary /6/./2/./1/, all the submatrices A /(/1 /: k/;; /1/: k /)/(/1 / k / n /) of the matrix A are p ositiv e de/nite/, and/, therefore/, regularmatrices/, and prop osition /6/./1/./1 implies the existence of the LD M Tdecom/-p osition/. T aking X /= L /; Tin prop osition /6/./2/./1/, w e /nd that the matrixB /= DM TL /; T/= L /; /1AL /; Tis p ositiv e de/nite/. Since the matrix M TL /; Tis an upp er triangular matrixwith the unit diagonal/, the matrices B and D ha v e the same leading diagonaland the elemen ts on it m ust b e p ositiv e/, pro vided that B is p ositiv e de/nite/./2Prop osition /6/./2/./2 /( Cholesky factorization /)/. If the matrix A /2 R n / nissymmetric and p ositiv e de/nite/, then there exists exactly one lo w er triangularmatrix G with the p ositiv e leading diagonal suc h thatA /= GG T/: /(/5/)Pr o of/. In virtue of prop osition /6/./1/./2/, there exist and are uniquely de/nedthe lo w er triangular matrix L with the unit diagonal and the diagonal matrixD /= diag /( d/1 /;;/:/:/: /;;dn /)s u c h that the decomp osition /(/3/) holds/, i/.e/./, A /= LD L T/:The corollary /6/./2/./2 pro vides that elemen ts dk of the matrix D are p ositiv e/.Therefore/, the matrixG /= L p D /= L /
diag /( q d/1 /;;/:/:/: /;; q dn /) /2 R n / nis a lo w er triangular matrix with the p ositiv e leading diagonal/, and equalit y/(/5/) holds/. The uniqueness of the the factorization follo ws from the uniquenessof the decomp osition /(/3/)/. /2The factorization /(/5/) is kno wn as the Cholesky factorization /. The matrixG is called the Cholesky triangular matrix of the matrix A /. T o solv e thesystem of equationsA x /= bha ving the symmetric and p ositiv e de/nite matrix A /, one has to /nd theCholesky triangular matrix of the matrix A /. Secondly /, one has to solv e thesystem with the triangular matrixG y /= b /:/1/4/6 Thirdly /, one has to solv e the systemG Tx /= y /:The Cholesky factorization can b e found step b y step/.Prop osition /6/./2/./3/. If the matrix A /2 R n / nis symmetric and p ositiv ede/nite/, then/, denotingA /= /"/ v Tv B /#/;;the matrix A can be expressed b yA /= /"/ v Tv B /#/= /"/ /0 Tv /=/ In /; /1 /#/"/1 /0 T/0 B /; vv T/=/ /#/"/ v T/=//0 In /; /1 /#/;; /(/6/)where / /= p //: The matrix B /; vv T/=/ is p ositiv e de/nite/. IfB /; vv T/=/ /= G/1 G T/1 /;;then A /= GG T/;; whereG /= /"/ /0 Tv /=/ G/1 /#/:Pr o of/. Let us c hec k the accurancy of the decomp osition /(/6/)/:/"/ /0 Tv /=/ In /; /1 /#/"/1 /0 T/0 B /; vv T/=/ /#/"/ v T/=//0 In /; /1 /#/=/= /"/ /0 Tv /=/ B /; vv T/=/ /#/"/ v T/=//0 In /; /1 /#/= /"/ /2v Tv vv T/=/ /2/+ B /; vv T/=/ /#/=/= /"/ v Tv B /#/= A/:IfX /= /"/1 /; v T/=//0 In /; /1 /#/;;thenX TAX /= /"/1 /0 T/; v /=/ In /; /1 /#/"/ v Tv B /#/"/1 /; v T/=//0 In /; /1 /#/=/1/4/7 /= /"/ v T/0 B /; vv T/=/ /#/"/1 /; v T/=//0 In /; /1 /#/= /"/ /0 T/0 B /; vv T/=/ /#/:Since the matrix A is p ositiv e de/nite and the column/-v ector system of thematrix X is linearly indep enden t/, prop osition /6/./2/./1 implies the p ositiv e de//-niteness of the matrix/"/ /0 T/0 B /; vv T/=/ /#and from corollary /6/./2/./1 it follo ws that the matrix B /; vv T/=/ is lik ewisep ositiv e de/nite/. So w e can/, analogously to the partition of the matrix Ain to blo c ks/, decomp ose the matrix B /; vv T/=/ in to blo c ks/, etc/.Example /6/./2/./2/. Let us /nd the LU /, LD M T/, LD L Tand Choleskyfactorizations of the matrixA /= /2/6/4 /1 /2 /0/2 /8 /4/0 /4 /1/3 /3/7/5 /:The principal minors of the matrix A are nonzero/. W e /ndA /= /2/6/4 /1 /2 /0/2 /8 /4/0 /4 /1/3 /3/7/5 l/2/1 /=/2/!l/3/1 /=/0 /2/6/4 /1 /2 /0/0 /4 /4/0 /4 /1/3 /3/7/5 l/3/2 /=/1/! /2/6/4 /1 /2 /0/0 /4 /4/0 /0 /9 /3/7/5 /= U/;;andL /= /2/6/4 /1 /0 /0/2 /1 /0/0 /1 /1 /3/7/5and alsoA /= LU /= /2/6/4 /1 /0 /0/2 /1 /0/0 /1 /1 /3/7/5 /2/6/4 /1 /2 /0/0 /4 /4/0 /0 /9 /3/7/5 /:Kno wing the LU factorization of the matrix A /, w e will /nd the LD M Tdecomp osition/, LD L Tdecomp osition and Cholesky factorization of it/:A /= LD M T/= /2/6/4 /1 /0 /0/2 /1 /0/0 /1 /1 /3/7/5 /2/6/4 /1 /0 /0/0 /4 /0/0 /0 /9 /3/7/5 /2/6/4 /1 /2 /0/0 /1 /1/0 /0 /1 /3/7/5 /= LD L T/1/4/8 andA /= GG T/= /2/6/4 /1 /0 /0/2 /2 /0/0 /2 /3 /3/7/5 /2/6/4 /1 /2 /0/0 /2 /2/0 /0 /3 /3/7/5 /:Let us /nd the Cholesky factorization of the matrix A also step b y step usingthe algorithm giv en in prop osition /6/./2/./3/. Since at the /rst step//1 /=/1 /;; //1 /= p //1 /=/1 /;; v/1 /= /"/2/0 /#/;; B/1 /= /"/8 /4/4 /1/3 /#/;;thenB/1 /; v/1 v T/1 /=//1 /= /"/8 /4/4 /1/3 /#/; /"/2/0 /#h/2 /0 i/= /1/= /"/4 /4/4 /1/3 /#/:On the next step/,//2 /=/4 /;; //2 /= p /4/= /2 /;; v/2 /= h/4 i/;; B/2 /= h/1/3 iandB/2 /; v/2 v T/2 /=//2 /= h/1/3 i/; h/4 ih/4 iT/= /4/= h/9 i/= h/3 ih/3 iT/:Therefore/,G/2 /= h/3 iandG/1 /= /"//2 /0v/2 /=//2 G/2 /#/= /"/2 /0/2 /3 /#andG /= /"//1 /0v/1 /=//1 G/1 /#/= /2/6/4 /1 /0 /0/2 /2 /0/0 /2 /3 /3/7/5 /:Problem /6/./2/./2/. /Find the Cholesky factorization of the p ositiv e de/nitematrixA /= /2/6/6/6/4 /1 /2 /; /1 /2/2 /8 /6 /0/; /1 /6 /2/1 /; /2/2 /0 /; /2 /2/5 /3/7/7/7/5 /:/1/4/9 Problem /6/./2/./3/. /Solv e the system of equations A x /= b /;; whereA /= /2/6/4 /1 /; /1 /1/; /1 /1/0 /; /1/0/1 /; /1/0 /1/4 /3/7/5 /^ b /= /2/6/4 /2/; /2/6 /3/7/5 /;;when the Cholesky factorization of the matrix A is giv enA /= GG T/^ G /= /2/6/4 /1 /0 /0/; /1 /3 /0/1 /; /3 /2 /3/7/5 /:/2/./6/./3 P ositiv e Semide/nite MatricesDe/nition /6/./3/./1/. A matrix A /2 R n / nis called a p ositive semide/nitematrix if/8 x /2 R n/) x TA x / /0 /:Example /6/./3/./1/. The matrixA /= /"/1 /1/1 /1 /#is p ositiv e semide/nite since for /8 x /= h//1 //2 iT/2 R /2x TA x /= h//1 //2 i /"/1 /1/1 /1 /#/"//1//2 /#/=/= h//1 //2 i /"//1 /+ //2//1 /+ //2 /#/=/( //1 /+ //2 /) /2/ /0 /;;and in the case //1 /= /; //2 /^ //1 /6/=/0 w e see that x TA x /=/0 /;; but x /6/= /0 /;; i/.e/./, thematrix A is a p ositiv e semide/nite matrix/, but it is not p ositiv e de/nite/.Problem /6/./3/./1/. /Sho w that the matrixA /= /2/6/4 /1 /2 /3/2 /4 /6/3 /6 /9 /3/7/5/1/5/0 is p ositiv e semide/nite/.Prop osition /6/./3/./1/. If A /2 R n / nis a symmetric p ositiv e semide/nitematrix/, thenj aij j/ p aii ajj /;; /(/7/)aii /=/0 /) A /( i/;; /:/) /= A /(/: /;;i /)/= /0 /;; /(/8/)j aij j/ /( aii /+ ajj /) /= /2 /(/9/)andmaxi/;; j j aij j /=maxi aii /: /(/1/0/)Pr o of/. Let / /2 R and x /= ei /+ / ej /2 R n/: Since the matrix A is p ositiv esemide/nite and symmetric/, then/0 / x TA x /= h/0 T/1 /0 T/ /0 T i /2/6/6/4 a/1/1 /
/
/
ann/././. /././. /././.an /1 /
/
/
ann /3/7/7/5 /2/6/6 /6 /6/6/6/4 /0/1/0//0 /3/7/7 /7 /7/7/7/5 /=/= h/0 T/1 /0 T/ /0 T i /2/6/6/4 a/1 i /+ /a/1 j/././.ani /+ /anj /3/7/7/5 /= aii /+ /aij /+ /aji /+ / /2ajj /;;andaii /+/2 /aij /+ / /2ajj / /0 /: /(/1/1/)Condition /(/1/1/) is satis/ed exactly whena /2ij /; aii ajj / /0 /;;from whic h/, in its turn/, it follo ws /(/7/)/, and from it /(/8/)/. Fixing in inequalit y/(/1/1/) / /= / /1 and taking in to consideration the symmetry of the matrix A /,w e getaii /+ ajj //; /2 aij /;;aii /+ ajj / /2 aij /;;and assertions /(/9/) and /(/1/0/)/. /2Problem /6/./3/./2/. Sho w that the algorithm of the Cholesky factorizationA /= GG Tis applicable /(with small c hanges/) also to the symmetric p ositiv esemide/nite matrix A /./1/5/1 /2/./6/./4 P olar Decomp osition of a Matrix and Metho d of SquareRo otsProp osition /6/./4/./1 /( on the r e duc e d singular value de c omp osition of a ma/-trix /)/. If the matrix A /2 R m / n/( m / n /) has the singular v alue decomp ositionA /= U / V T/;; where U /2 R m / mand V /2 R n / nare orthogonal matrices and//= diag /( //1 /;;/:/:/: /;;/n /) /2 R m / n/;; then the r e duc e d singular value de c omp ositionof this matrix A isA /= U/1 //1 V T/;;where U/1 /= U /(/: /;; /1/: n /) and //1 /= //(/1 /: n/;; /:/) /:Pr o of/. If one uses the represen tation of the matrices U and V b y thecolumn/-v ectorsU /= hu/1 /
/
/
um iandV /= hv/1 /
/
/
vn i/;;thenA /= U / V T/= hu/1 /
/
/
um i /2/6/6/6/6/4 //1 /
/
/
/0/./. /. /././. /./. /./0 /
/
/
/n/0 /
/
/
/0 /3/7/7/7/7/5 /2/6/6/4 v T/1/./. /.v Tn /3/7/7/5 /=/= hu/1 /
/
/
um i /2/6/6 /6 /6/4 //1 v T/1/./././n v Tn/0 /3/7/7 /7 /7/5 /= //1 u/1 v T/1 /+ /:/:/: /+ //1 un v Tn /+/0andA /= U/1 //1 V T/= hu/1 /
/
/
un i /2/6/6/4 //1 /
/
/
/0/././. /././. /./././0 /
/
/
/n /3/7/7/5 /2/6/6/4 v T/1/././.v Tn /3/7/7/5 /=/= //1 u/1 v T/1 /+ /:/:/: /+ //1 un v Tn /:Example /6/./4/./1/. Find the reduced singular v alue decomp osition of thematrixA /= /2/6/4 /1 /1/0 /1/1 /0 /3/7/5 /2 R /3 / /2/:/1/5/2 The singular v alue decomp osition A /= U / V Tof the matrix A w as foundin example /3/./3/./1/. It isA /= /2/6/4 p /6 /= /3 /0 p /3 /= /3p /6 /= /6 /; p /2 /= /2 /; p /3 /= /3p /6 /= /6 p /2 /= /2 /; p /3 /= /3 /3/7/5 /2/6/4 p /3 /0/0 /1/0 /0 /3/7/5 /"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/:According to prop osition /6/./4/./1/, the reduced singular v alue decomp osition hasthe formA /= U/1 //1 V T/;;where U/1 /= U /(/: /;; /1/: n /) and //1 /= //(/1 /: n/;; /:/) /;; i/.e/./,A /= /2/6/4 p /6 /= /3 /0p /6 /= /6 /; p /2 /= /2p /6 /= /6 p /2 /= /2 /3/7/5 /"p /3 /0/0 /1 /#/"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/:Prop osition /6/./4/./2/. If the matrix A /2 R m / nhas the reduced singularv alue decomp ositionA /= U/1 //1 V T/;;then the matrix A can b e written in the formA /= ZP /;; /(/1/2/)where Z /= U/1 V T/2 R m / nis a matrix with orthogonal columns and P /=V //1 V Tis a symmetric p ositiv e semide/nite matrix/.Pr o of/. Since A /= U/1 //1 V T/;; thenA /= U/1 /( V TV /)//1 V T/=/( U/1 V T/)/( V //1 V T/)/= ZP /:Let us c hec k the correctness of the assertion of the prop osition/. Firstly /, Z isa matrix with orthonormal columns sinceZ TZ /=/( U/1 V T/) T/( U/1 V T/)/= V /( U T/1 U/1 /) V T/= VV T/= I/:Secondly /, P /= V //1 V Tis a p ositiv e semide/nite matrix sincex TP x /= x TV //1 V Tx /=/( V Tx /) T//1 /( V Tx /)/= nXi /=/1 /i / /2i / /0/( /8 x /2 R n/) /;;/1/5/3 where /i /= Pnk /=/1 vki /k /: /2De/nition /6/./4/./1/. The factorization of the matrix A /2 R m / nin the form/(/1/2/) is called the p olar de c omp osition /.Example /6/./4/./2/. Find the p olar decomp osition of the matrixA /= /2/6/4 /1 /1/0 /1/1 /0 /3/7/5 /2 R /3 / /2/:In example /6/./4/./1 the reduced singular v alue decomp osition A /= U/1 //1 V Tof the matrix A w as found/. Let us /nd the factors Z and P o ccuring in thep olar decomp osition of the matrix A /:Z /= U/1 V T/= /2/6/4 p /6 /= /3 /0p /6 /= /6 /; p /2 /= /2p /6 /= /6 p /2 /= /2 /3/7/5 /"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/=/= /2/6/4 /1 /3 p /3 /1 /3 p /3/1 /6 p /3 /; /1 /2 /1 /6 p /3/+ /1 /2/1 /6 p /3/+ /1 /2 /1 /6 p /3 /; /1 /2 /3/7/5 /;;P /= V //1 V T/= /"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/"p /3 /0/0 /1 /#/"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/= /"/1 /2 p /3/+ /1 /2 /1 /2 p /3 /; /1 /2/1 /2 p /3 /; /1 /2 /1 /2 p /3/+ /1 /2 /#/:Hence the p olar decomp osition of the matrix A isA /= ZP /= /2/6/4 /1 /3 p /3 /1 /3 p /3/1 /6 p /3 /; /1 /2 /1 /6 p /3/+ /1 /2/1 /6 p /3/+ /1 /2 /1 /6 p /3 /; /1 /2 /3/7/5 /"/1 /2 p /3/+ /1 /2 /1 /2 p /3 /; /1 /2/1 /2 p /3 /; /1 /2 /1 /2 p /3/+ /1 /2 /#/:Problem /6/./4/./1/. /Find the p olar decomp osition of the matrixA /= /2/6/4 /1 /; /1/; /1 /1p /2 p /2 /3/7/5 /:/1/5/4 De/nition /6/./4/./2/. Let A /2 R m / n/: If the matrix X /2 R m / nsatis/es theequation X /2/= A/;; then the matrix X is the squar e r o ot of the matrix A/:Prop osition /6/./4/./3/. IfA /= GG Tis the Cholesky factorization of the symmetric p ositiv e semide/nite matrixA /2 R n / nandG /= U / V Tis the singular v alue decomp osition of the matrix G andX /= U / U T/;;thenX /2/= A/;;i/.e/./, the matrix X is the square ro ot of the matrix A/;; where X is a symmetricp ositiv e semide/nite matrix/. Only one suc h X exists/.Pr o of/. W e /ndA /= GG T/=/( U / V T/)/( U / V T/) T/= U / V TV / U T/= U / /2U T/=/= U //( U TU /)/ U T/=/( U / U T/)/( U / U T/)/= X /2/:Sho w that the matrix X is a uniquely de/ned symmetric p ositiv e semide/nitematrix/! /2Example /6/./4/./3/. Let us /nd the square ro ot of the matrixA /= /"/1 /1/1 /1 /#/:The matrix A is symmetric and p ositiv e semide/nite /(see example /6/./3/./1/)/,andA /= GG T/= /"/1 /0/1 /0 /#/"/1 /1/0 /0 /#/:SinceG /= U / V T/= /"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/"p /2 /0/0 /0 /#/"/1 /0/0 /1 /#/;;thenX /= U / U T/=/1/5/5 /= /"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/"p /2 /0/0 /0 /#/"p /2 /= /2 p /2 /= /2p /2 /= /2 /; p /2 /= /2 /#/=/= /"/1 /2 p /2 /1 /2 p /2/1 /2 p /2 /1 /2 p /2 /#/:Problem /6/./4/./2/. /Find the square ro ot of the matrixA /= /2/6/4 /2 /1 /0/1 /2 /0/0 /0 /4 /3/7/5 /:/2/./6/./5 Systems with Band MatricesIn man y applications the matrix A of the system of equations A x /= b is aband matrix/, i/.e/./, the unkno wn quanlit y /i app ears with a nonzero co e/cien tonly in the i /-th equation and some /\neigh b ouring/" equations/.Prop osition /6/./5/./1/. Let A /= LU be the LU factorization of the bandmatrix A /2 R n / n/. If the upp er band width of the matrix A is q and thelo w er band width is p/;; then the matrix U has the upp er band width q andthe matrix L has the lo w er band width p/:Pr o of/. W e will pro v e it b y induction/. In the case n /= /1 /;; this assertionis v alid/. Let us sho w the admissibilit y of the step of induction/. Let theprop osition b e correct for an /( n /; /1/) / /( n /; /1/) matrix A /. Let the matrix Abe giv en in the formA /= /"/ w Tv B /#/:The follo wing equalit yi s v alid/:A /= /"/1 /0 Tv /= / In /; /1 /#/"/1 /0 T/0 B /; vw T/=/ /#/"/ w T/0 In /; /1 /#/:Since in the v ectors v and w at the most only the /rst p and q co ordinates aredi/eren t from zero/, then the matrix B /; vw T/=/ has the upp er band widthp and the lo w er band width q /. The matrix B /; vw T/=/ i sa/( n /; /1/) / /( n /; /1/)/1/5/6 matrix/, and/, hence B /; vw T/=/ /= L/1 U/1 /;; where U/1 has the upp er band widthq and L/1 has the lo w er band width p /. The matricesL /= /"/1 /0 Tv /=/ L/1 /#andU /= /"/ w T/0 U/1 /#ha v e the band width p and q /, resp ectiv ely /, and A /= LU/: /2Problem /6/./5/./1/. Find for the LU factorization of the matrix giv en inexample /6/./2/./1 the upp er and lo w er band width for the matrices A/;; L and U /./2/./6/./6 Blo c k SystemsLet us consider the system in the form/2/6/6/6 /6 /6 /6/6/6/4 D/1 F/1 /
/
/
/0E/1 D/2 /././. /./././././. /././. /./././././. /././.Dn /; /1 Fn /; /1/0 /
/
/
En /; /1 Dn /3/7/7/7 /7 /7 /7/7/7/5 /2/6/6 /6 /6 /6 /6/6/4 x/1x/2/././.xn /; /1xn /3/7/7 /7 /7 /7 /7/7/5 /= /2/6/6 /6 /6 /6 /6/6/4 b/1b/2/././.bn /; /1bn /3/7/7 /7 /7 /7 /7/7/5 /;; /(/1/3/)where Di /;; Ei /, Fi /2 R q / qand xi /, bi /2 R q/: If w e represen t the matrix A inthe formA /= /2/6/6 /6/6/6 /6 /6 /6/4 I /
/
/
/0L/1 I /./. /./././. /./././././. /././.I/0 /
/
/
Ln /; /1 I /3/7/7 /7/7/7 /7 /7 /7/5 /2/6/6 /6/6/6 /6 /6 /6/4 U/1 F/1 /
/
/
/0U/2 /././. /./. /./././. /./././././. Un /; /1 Fn /; /1/0 /
/
/
Un /3/7/7 /7/7/7 /7 /7 /7/5 /;;/1/5/7 thenA /= /2/6/6 /6 /6/6/6 /6 /6 /6 /6/6/4 U/1 F/1 /
/
/
/0L/1 U/1 L/1 F/1 /+ U/2 F/2 /././.L/2 U/2 L/2 F/2 /+ U/3 F/3L/3 U/3 /././. /./././././. /././. /././.Fn /; /1/0 /
/
/
Ln /; /1 Un /; /1 Ln /; /1 Fn /; /1 /+ Un /3/7/7 /7 /7/7/7 /7 /7 /7 /7/7/5 /:Let us /nd step b y step the blo c ks Li and Ui /:U/1 /= D/1 /! solv e L/1 U/1 /= E/1 /!/! U/2 /= D/2 /; L/1 F/1 /! solv e L/2 U/2 /= E/2 /!/
/
/
/!/! Un /; /1 /= Dn /; /1 /; Ln /; /2 Fn /; /2 /! solv e Ln /; /1 Un /; /1 /= En /; /1 /!/! Un /= Dn /; Ln /; /1 Fn /; /1 /:T o solv e system /(/1/3/)/, one m ust /rst solv e the system/2/6/6 /6 /6 /6/6/6 /6/4 I /
/
/
/0L/1 I /./././././. /./././././. /././.I/0 /
/
/
Ln /; /1 I /3/7/7 /7 /7 /7/7/7 /7/5 /2/6/6 /6 /6 /6/6/6/4 y/1y/2/././.yn /; /1yn /3/7/7 /7 /7 /7/7/7/5 /= /2/6/6 /6 /6 /6/6/6/4 b/1b/2/././.bn /; /1bn /3/7/7 /7 /7 /7/7/7/5 /:W e /nd thaty/1 /= b/1 /! L/1 y/1 /+ y/2 /= b/2 /! y/2 /= b/2 /; L/1 y/1 /! /
/
/
/!/! Li /; /1 yi /; /1 /+ yi /= bi /! yi /= bi /; Li /; /1 yi /; /1 /! /
/
/
/!/! Ln /; /1 yn /; /1 /+ yn /= bn /! yn /= bn /; Ln /; /1 yn /; /1 /:Secondly /, w e ha v e to solv e the system/2/6/6 /6 /6/6/6/6/6/4 U/1 F/1 /
/
/
/0U/2 /././. /./. /./././. /././././. /. Un /; /1 Fn /; /1/0 /
/
/
Un /3/7/7 /7 /7/7/7/7/7/5 /2/6/6 /6 /6/6/6/6/4 x/1x/2/./. /.xn /; /1xn /3/7/7 /7 /7/7/7/7/5 /= /2/6/6 /6 /6/6/6/6/4 y/1y/2/./. /.yn /; /1yn /3/7/7 /7 /7/7/7/7/5 /:/1/5/8 Example /6/./6/./1/. /Let us solv e the system of equations A x /= b /;; whereA /= /2/6/6/6/6 /6 /6 /6/6/4 /1 /; /1 /2 /1 /0 /0/1 /0 /1 /0 /0 /0/0 /2 /2 /1 /; /1 /1/1 /; /1 /; /1 /1 /2 /1/0 /0 /1 /1 /1 /1/0 /0 /; /1 /1 /2 /; /1 /3/7/7/7/7 /7 /7 /7/7/5 /^ b /= /2/6/6/6/6 /6 /6 /6/6/4 //1//2//3//4//5//6 /3/7/7/7/7 /7 /7 /7/7/5 /= /2/6/6/6/6 /6 /6 /6/6/4 /1/1/; /4/3/0/1 /3/7/7/7/7 /7 /7 /7/7/5 /:This is a blo c k system giv en b y relation /(/1/3/) sinceA /= /2/6/4 D/1 F/1 /0E/1 D/2 F/2/0 E/2 D/3 /3/7/5 /;;where Di /;; Ei /;; Fi /2 R /2 / /2andD/1 /= /"/1 /; /1/1 /0 /#/^ F/1 /= /"/2 /1/1 /0 /#/^ E/1 /= /"/0 /2/1 /; /1 /#/^ D/2 /= /"/2 /1/; /1 /1 /#/^ F/2 /= /"/; /1 /1/2 /1 /#/^ E/2 /= /"/1 /1/; /1 /1 /#/^ D/3 /= /"/1 /1/2 /; /1 /#/:W e will express the matrix A in the formA /= LU /= /2/6/4 I/2 /0 /0L/1 I/2 /0/0 L/2 I/2 /3/7/5 /2/6/4 U/1 F/1 /0/0 U/2 F/2/0 /0 U/3 /3/7/5 /=/= /2/6/4 U/1 F/1 /0L/1 U/1 L/1 F/1 /+ U/2 F/2L/2 U/2 L/2 F/2 /+ U/3 /3/7/5 /:No w w e /ndU/1 /= D/1 /= /"/1 /; /1/1 /0 /#/;;L/1 U/1 /= E/1 /) L/1 /= /"/; /2 /2/1 /0 /#/;;L/1 F/1 /+ U/2 /= D/2 /) U/2 /= /"/4 /3/; /3 /0 /#/;;/1/5/9 L/2 U/2 /= E/2 /) L/2 /= /"/1 /= /3 /1 /= /9/1 /= /3 /7 /= /9 /#/;;L/2 F/2 /+ U/3 /= D/3 /) U/3 /= /"/1/0 /= /9 /5 /= /9/7 /= /9 /; /1/9 /= /9 /#andA /= /2/6/6/6 /6 /6 /6/6/6/4 /1 /0 /0 /0 /0 /0/0 /1 /0 /0 /0 /0/; /2 /2 /1 /0 /0 /0/1 /0 /0 /1 /0 /0/0 /0 /1 /= /3 /1 /= /9 /1 /0/0 /0 /1 /= /3 /7 /= /9 /0 /1 /3/7/7/7 /7 /7 /7/7/7/5 /2/6/6/6 /6 /6 /6/6/6/4 /1 /; /1 /2 /1 /0 /0/1 /0 /1 /0 /0 /0/0 /0 /4 /3 /; /1 /1/0 /0 /; /3 /0 /2 /1/0 /0 /0 /0 /1/0 /= /9 /5 /= /9/0 /0 /0 /0 /7 /= /9 /; /1/9 /= /9 /3/7/7/7 /7 /7 /7/7/7/5 /:T o /nd a solution of the system A x /= b /,w e shall solv et w o systems L y /= band U x /= y /: The system L y /= b can be expressed in the form/2/6/4 I/2 /0 /0L/1 I/2 /0/0 L/2 I/2 /3/7/5 /2/6/4 y/1y/2y/3 /3/7/5 /= /2/6/4 b/1b/2b/3 /3/7/5 /;;whereb/1 /= /"//1//2 /#/= /"/1/1 /#/^ b/2 /= /"//3//4 /#/= /"/; /4/3 /#/^/^ b/3 /= /"//5//6 /#/= /"/0/1 /#/;;andy/1 /= b/1 /= /"/1/1 /#/^ y/2 /= b/2 /; L/1 y/1 /= /"/; /4/2 /#/^/^ y/3 /= b/3 /; L/2 y/2 /:Solving the system U x /= y /;; whic h can be giv en in the form/2/6/4 U/1 F/1 /0/0 U/2 F/2/0 /0 U/3 /3/7/5 /2/6/4 x/1x/2x/3 /3/7/5 /= /2/6/4 y/1y/2y/3 /3/7/5 /;;w e obtainx/3 /= U /; /1/3 y/3 /= /"/1/0 /#/^ x/2 /= U /; /1/2 /( y/2 /; F/2 x/3 /)/= /"/0/; /1 /#/^/1/6/0 /^ x/1 /= U /; /1/1 /( y/1 /; F/1 x/2 /)/= /"/1/; /1 /#/:Th us/,x /= /2/6/4 x/1x/2x/3 /3/7/5 /= h/1 /; /1 /0 /; /1 /1 /0 iT/:/2/./6/./7 Solution of the Systems of Equations b y QR Metho dLet us consider the systemA x /= b /;; /(/1/4/)where A /= QR is a QR factorization of the regular matrix A /2 R n / n/,w h i l eQ /2 R n / nis an orthogonal matrix and R /2 R n / nia on upp er triangularmatrix/. Substituting in /(/1/4/) the matrix A b y its QR factorization/, w e getQR x /= b /: /(/1/5/)Multiplying the b oth sides of equalit y /(/1/5/) on the left b y the matrix Q T/;; w e/ndR x /= Q Tb /: /(/1/6/)System /(/1/6/) has an upp er triangular matrix R/: F rom the regularit y of thematrix A it follo ws the regularit y of the matrix R /. Hence system /(/1/6/) isuniquely solv able/. F or this the substitution giv en in prop osition /1/./1/./2 will b eused bac kw ards/.Example /6/./7/./1/. Let us solv e the system/2/6/4 /2 /0 /1/6 /2 /0/; /3 /; /1 /; /1 /3/7/5 x /= /2/6/4 /1/0/1 /3/7/5 /(/1/7/)using the QR metho d/.In example /2/./3/./2 the QR factorization of the matrix of the system/2/6/4 /2 /0 /1/6 /2 /0/; /3 /; /1 /; /1 /3/7/5 /= p /5 /3/5 /2/6/4 /; /2 p /5 /; /1/5 /0/; /6 p /5 /4 /; /7/3 p /5 /; /2 /; /1/4 /3/7/5 /2/6/6/4 /; /7 /; /1/5 /7 /; /5 /7/0 /2 p /5 /7 /; /1/3 p /5 /3/5/0 /0 /2 p /5 /5 /3/7/7/5/1/6/1 w as found/. W e resp ect system /(/1/7/) in form /(/1/6/)/:/2/6/4 /; /7 /; /1/5 /7 /; /5 /7/0 /2 /7 p /5 /; /1/3 /3/5 p /5/0 /0 /2 /5 p /5 /3/7/5 x /= p /5 /3/5 /2/6/4 /; /2 p /5 /; /6 p /5 /3 p /5/; /1/5 /4 /; /2/0 /; /7 /; /1/4 /3/7/5 /2/6/4 /1/0/1 /3/7/5 /;;i/.e/./,/2/6/4 /; /7 /; /1/5 /7 /; /5 /7/0 /2 /7 p /5 /; /1/3 /3/5 p /5/0 /0 /2 /5 p /5 /3/7/5 x /= /2/6/4 /1 /7/; /1/7 /3/5 p /5/; /2 /5 p /5 /3/7/5 /:W e solv e the obtained system with the upp er triangular matrix using thebac kw ards substitution/. The result is x /= h/1 /; /3 /; /1 iT/:Let us consider the solving of system /(/1/4/)/, where A /= QR is a QRfactorization of the regular matrix A /2 R m / n/( m / n /)/, where Q /2 R m / misan orthogonal matrix and R /2 R n / nis an upp er triangular matrix/, b y theleast/-squares metho d/. LetQ TA /= R /= /"R/1/0 /#andQ Tb /= /"cd /#/;;where R/1 /2 R n / n/;; c /2 R nand d /2 R m /; n/: W e /ndk A x /; b k /2/2 /= / / / Q TA x /; Q Tb / / / /2/2 /= / / / / / /"R/1/0 /#x /; /"cd /# / / / / / /2/2 /=/= / / / / / /"R/1 x /; c/0 /; d /# / / / / / /2/2 /= k R/1 x /; c k /2/2 /+ k d k /2/2 /:Since the quan tit y k d k /2/2 is a constan t/, w e can minimize only the quan tit yk R/1 x /; c k /2/2 /;;and the minimal v alue of it is /0/. Really /, from the condition dim A /= n itfollo ws that the matrix R/1 is regular/. Hence the systemR/1 xLS /= c /;;/1/6/2 where the sym bo l xLS i denotes the least/-squares solution of system /(/1/4/)/, isuniquely solv able/.Example /6/./7/./2/. Let us /nd the least squares solution of the system/2/6/6 /6 /6/6/6/4 /1 /0 /0/0 /1 /0/0 /0 /; /1/1 /0 /0/0 /; /1 /0 /3/7/7 /7 /7/7/7/5 x /= /2/6/6 /6 /6/6/6/4 /1/0/1/0 /1 /3/7/7 /7 /7/7/7/5using the QR metho d/.Using the soft w are pac k age /\Maple/"/, w e obtain the QR factorization ofthe matrix of the system/2/6/6/6 /6 /6 /6/4 /1 /0 /0/0 /1 /0/0 /0 /; /1/1 /0 /0/0 /; /1 /0 /3/7/7/7 /7 /7 /7/5 /= /2/6/6/6 /6 /6 /6/4 /; /1 /= p /2 /0 /0 /; /1 /= p /2 /0/0 /; /1 /= p /2 /0 /0 /1 /= p /2/0 /0 /; /1 /0 /0/; /1 /= p /2 /0 /0 /1 /= p /2 /0/0 /1 /= p /2 /0 /0 /1 /= p /2 /3/7/7/7 /7 /7 /7/5 /2/6/6/6 /6 /6 /6/4 /; p /2 /0 /0/0 /; p /2 /0/0 /0 /1/0 /0 /0/0 /0 /0 /3/7/7/7 /7 /7 /7/5 /:F rom this factorization it arrears thatR/1 /= /2/6/4 /; p /2 /0 /0/0 /; p /2 /0/0 /0 /1 /3/7/5 /:T o get the v ector c /,w e /ndQ Tb /= /2/6/6 /6 /6 /6/6/4 /; /1 /2 p /2 /0 /0 /; /1 /2 p /2 /0/0 /; /1 /2 p /2 /0 /0 /1 /2 p /2/0 /0 /; /1 /0 /0/; /1 /2 p /2 /0 /0 /1 /2 p /2 /0/0 /1 /2 p /2 /0 /0 /1 /2 p /2 /3/7/7 /7 /7 /7/7/5 /2/6/6 /6 /6 /6/6/4 /1/0/1/0/1 /3/7/7 /7 /7 /7/7/5 /= /2/6/6 /6 /6 /6/6/4 /; /1 /2 p /2/1 /2 p /2/; /1/; /1 /2 p /2/1 /2 p /2 /3/7/7 /7 /7 /7/7/5 /:Hencec /= h/; /1 /2 p /2 /1 /2 p /2 /; /1 iT/:W e get for the concrete form of the system R/1 xLS /= c/2/6/4 /; p /2 /0 /0/0 /; p /2 /0/0 /0 /1 /3/7/5 xLS /= /2/6/4 /; /1 /2 p /2/1 /2 p /2/; /1 /3/7/5 /;;/1/6/3 from whic h it follo ws thatxLS /= h/1 /2 /; /1 /2 /; /1 iT/:Example /6/./7/./3/. /Let us /nd the least/-squares solution of the system/2/6/4 /1 /1/2 /3/2 /1 /3/7/5 /"//1//2 /#/= /2/6/4 /1/1/1 /3/7/5 /:In example /2/./3/./3 it w as found the QR factorization of the matrix of thesystem/:/2/6/4 /1 /1/2 /3/2 /1 /3/7/5 /= /2/6/4 /1 /3 /0 /2 /3 p /2/2 /3 /1 /2 p /2 /; /1 /6 p /2/2 /3 /; /1 /2 p /2 /; /1 /6 p /2 /3/7/5 /2/6/4 /3 /3/0 p /2/0 /0 /3/7/5 /= QR /:Omitting the last ro w of zeros in the matrix R /,w e getR/1 /= /"/3 /3/0 p /2 /#/:No w w e /ndQ Tb /= /2/6/4 /1 /3 /2 /3 /2 /3/0 /1 /2 p /2 /; /1 /2 p /2/2 /3 p /2 /; /1 /6 p /2 /; /1 /6 p /2 /3/7/5 /2/6/4 /1/1/1 /3/7/5 /= /2/6/4 /5 /3/0/1 /3 p /2 /3/7/5T aking the /rst t w o comp onen ts of this v ector /(the matrix R/1 has t w or o ws/)/,w e obtainc /= /"/5 /3/0 /#/:W e get the least/-squares solution of the initial system from R/1 xLS /= c /;; i/.e/./,/"/3 /3/0 p /2 /#xLS /= /"/5 /3/0 /#/) xLS /= /"/5 /9/0 /#/:Problem /6/./7/./1/. /Solv e the system of equations/2/6/4 /1/2 /; /3 /1/; /3 /1 /2/4 /; /4 /3 /; /1 /3/7/5 x /= /2/6/4 /; /2/2/1 /3/7/5/1/6/4 kno wing the QR factorization of the system matrix/2/6/4 /1/2 /; /3 /1/; /3 /1 /2/4 /; /4 /3 /; /1 /3/7/5 /= /2/6/4 /1/2 /1/3 /; /5 /1/3 /0/; /3 /1/3 /; /3/6 /6/5 /; /5/2 /6/5/4 /1/3 /4/8 /6/5 /; /3/9 /6/5 /3/7/5 /2/6/4 /1/3 /; /1/3/3 /3/9 /2 /1/3/0 /; /5 /1/3 /; /2/9 /1/3/0 /0 /; /1 /3/7/5 /:Problem /6/./7/./2/. /Find the least/-squares solution of the system/2/6/4 /0 /2/1 /3/0 /2 /3/7/5 /"//1//1 /#/= /2/6/4 /; /3/4/2 /3/7/5 /:/2/./7 Iterativ e Solution of Systems of Equations/2/./7/./1 P o w ers of a Matrix and In v erse MatrixProp osition /7/./1/./1/. If F /2 R n / nand k F kp /< /1 /;; then I /; F is a regularmatrix and/( I /; F /) /; /1/= /1Xk /=/0 F k/;;while/ / / /( I /; F /) /; /1 / / / p / /1 /1 /;k F kp /:Pr o of/. If w e supp ose the con trary to the assertion that the matrix I /; Fis singular/, then there exists a nonzero v ector x /2 R nthat /( I /; F /) x /= /0 /;; i/.e/./,x /= F x and k x kp /= k F x kp /, k F kp / /1 /: Hence the matrix I /; F is a regularmatrix/. T o /nd the matrix /( I /; F /) /; /1/, w e consider the iden tit y/ nXk /=/0 F k /!/( I /; F /)/= I /; F n /+/1/:Since/ / / F k / / / p /k F k kp /^ k F kp /< /1 /) limk /!/1 F k/=/0 /;;then/ limn /!/1 nXk /=/0 F k /!/( I /; F /)/= I/;;/1/6/5 that implies/( I /; F /) /; /1/= limn /!/1 nXk /=/0 F k/= /1Xk /=/0 F kand/ / / /( I /; F /) /; /1 / / / p / nXk /=/0 k F k kp / /1 /1 /;k F kp /;;whic h w as to b e pro v ed/. /2Prop osition /7/./1/./2/. Let Q HAQ /= T /= D /+ N b e the Sc h ur factorizationof the matrix A /2 C n / n/, while D is a diagonal matrix and N is a strictlyupp er triangular matrix /(on the leading diagonal there are zeros/)/. Let /and / be resp ectiv ely the greatest and the least mo dulus eigen v alues of thematrix A /. If / / /0 /;; then for all k / /0/ / / A k / / / /2 / /(/1 /+ / /) n /; /1 / j / j /+ k N kF /1/+ / /!k/:If A is a regular matrix and the n um be r / is suc h that/(/1 /+ / /) j / j /> k N kF /;;then for all k / /0/ / / A /; k / / / /2 / /(/1 /+ / /) n /; /1 / /1 j / j/; k F kF /= /(/1 /+ / /) /!k/:Pr o of/. See Golub/, Loan /(/1/9/9/6/, pp /3/3/6/-/3/3/7/)/. /2The form ulaB /; /1/= A /; /1/; B /; /1/( B /; A /) A /; /1whic h is easily c hec k ed sho ws ho w the in v erse matrix c hanges when the matrixA is substituted b y the matrix B/: The mo di/cation of this form ula is aformula of Sherman/-Morrison/-Wo o dbury giv en in the follo wing prop osition/.Prop osition /7/./1/./3/. If A /2 R n / nand U/;; V /2 R n / k/;; while matrices Aand I /+ V TA /; /1U are regular/,/( A /+ UV T/) /; /1/= A /; /1/; A /; /1U /( I /+ V TA /; /1U /) /; /1V TA /; /1/:Pr o of/. See Golub/, Loan /(/1/9/9/6/, pp/. /5/0/)/. /2/1/6/6 /2/./7/./2 Jacobi/'s and Gauss/-Seidel Metho dLet A /2 C n / nand aii /6/= /0 /(i /= /1 /: n/)/. W e will consider the solution ofthe system of equationsA x /= b /(/1/)b y an iterativ e metho d/.De/nition /7/./2/./1/. The appr oximation or the appr oximate value of thesolution x of system /(/1/) is a v ector that in certain sense di/ers little fromthe v ector x /: Let us represen t system /(/1/) in the form/i /=/( /i /; nXj /=/1j /6/= i aij /j /) /=aii /( i /= /1/: n /) /:Jacobi/'s iterativ e pro cess is de/ned b y the algorithm/ /( k /+/1/)i /=/( /i /; nXj /=/1j /6/= i aij / /( k /)j /) /=aii /( i /= /1/: n /) /: /(/2/)The Gauss/-Seidel iterativ e pro cess is de/ned b y the algorithm/ /( k /+/1/)i /=/( /i /; i /; /1Xj /=/1 aij / /( k /+/1/)j /; nXj /= i /+/1 aij / /( k /)j /) /=aii /( i /= /1/: n /) /: /(/3/)In case of b oth Jacobi/'s and the Gauss/-Seidel iterativ e pro cesses the transitionfrom the appro ximation x /( k /)/= f / /( k /)i g of the solution of system /(/1/) to the nextappro ximation x /( k /+/1/)/= f / /( k /+/1/)i g can be describ ed using the matricesL /= /2/6/6 /6 /6 /6/4 /0 /0 /
/
/
/0a/2/1 /0 /./././0/././. /././. /././. /././.an /1 an /2 /
/
/
/0 /3/7/7 /7 /7 /7/5 /;; U /= /2/6/6 /6 /6 /6/4 /0 a/1/2 /
/
/
a/1 n/0 /0 /././.a/2 n/././. /././. /././. /./././0 /0 /
/
/
/0 /3/7/7 /7 /7 /7/5and D /= diag /( a/1/1 /;;/:/:/: /;;ann /) /;; while A /= L /+ D /+ U/: F or example/, Jacobi/'salgorithm can be represen ted asMJ x /( k /+/1/)/= NJ x /( k /)/+ b/;; /(/4/)/1/6/7 where MJ /= D and NJ /= /; /( L /+ U /) /: The Gauss/-Seidel algorithm /(/3/) can berepresen ted asMG x /( k /+/1/)/= NG x /( k /)/+ b/;; /(/5/)where MG /= D /+ L and NG /= /; U/:Example /7/./2/./1/. /Let us solv e the system A x /= b /;; /, whereA /= /2/6/4 /1 /1 /; /1/; /1 /3 /0/1 /0 /; /2 /3/7/5 /^ b /= /2/6/4 /0/2/; /3 /3/7/5b y Jacobi/'s metho d/.Let us represen t the matrix A in the formA /= L /+ D /+ U /= /2/6/4 /0 /0 /0/; /1 /0 /0/1 /0 /0 /3/7/5 /+ /2/6/4 /1 /0 /0/0 /3 /0/0 /0 /; /2 /3/7/5 /+ /2/6/4 /0 /1 /; /1/0 /0 /0/0 /0 /0 /3/7/5 /:W e will form the matrices Mj and Nj /:Mj /= D /= /2/6/4 /1 /0 /0/0 /3 /0/0 /0 /; /2 /3/7/5 /;; Nj /= /; /( L /+ U /)/= /2/6/4 /0 /; /1 /1/1 /0 /0/; /1 /0 /0 /3/7/5The algorithm of Jacobi/'s iterativ e pro cess can b e giv en in the formx /( k /+/1/)/= M /; /1j Nj x /( k /)/+ M /; /1j b /:SinceM /; /1j Nj /= /2/6/4 /1 /0 /0/0 /1 /= /3 /0/0 /0 /; /1 /= /2 /3/7/5 /2/6/4 /0 /; /1 /1/1 /0 /0/; /1 /0 /0 /3/7/5 /= /2/6/4 /0 /; /1 /1/1 /3 /0 /0/1 /2 /0 /0 /3/7/5andM /; /1j b /= /2/6/4 /1 /0 /0/0 /1 /= /3 /0/0 /0 /; /1 /= /2 /3/7/5 /2/6/4 /0/2/; /3 /3/7/5 /= /2/6/4 /0/2 /3/3 /2 /3/7/5 /;;thenx /( k /+/1/)/= /2/6/4 /0 /; /1 /1/1 /3 /0 /0/1 /2 /0 /0 /3/7/5 x /( k /)/+ /2/6/4 /0/2 /3/3 /2 /3/7/5 /:/1/6/8 If w e tak e for the initial appro ximation x /(/0/)/= h/0 /0 /0 iT/;; then w e shallha v ex /(/1/)/= /2/6/4 /0 /; /1 /1/1 /3 /0 /0/1 /2 /0 /0 /3/7/5 /2/6/4 /0/0/0 /3/7/5 /+ /2/6/4 /0/2 /3/3 /2 /3/7/5 /= /2/6/4 /0/: /6/6/6/6/7/1 /: /5 /3/7/5 /;;x /(/2/)/= /2/6/4 /0 /; /1 /1/1 /3 /0 /0/1 /2 /0 /0 /3/7/5 /2/6/4 /0/: /6/6/6/6/7/1 /: /5 /3/7/5 /+ /2/6/4 /0/2 /3/3 /2 /3/7/5 /= /2/6/4 /: /8/3/3/3/3/: /6/6/6/6/7/1 /: /5 /3/7/5 /;;/:x /(/3/)/= /2/6/4 /0 /; /1 /1/1 /3 /0 /0/1 /2 /0 /0 /3/7/5 /2/6/4 /: /8/3/3/3/3/: /6/6/6/6/7/1 /: /5 /3/7/5 /+ /2/6/4 /0/2 /3/3 /2 /3/7/5 /= /2/6/4 /: /8/3/3/3/3/: /9/4/4/4/4/1 /: /9/1/6/7 /3/7/5 /;;x /(/4/)/= /2/6/4 /0 /; /1 /1/1 /3 /0 /0/1 /2 /0 /0 /3/7/5 /2/6/4 /: /8/3/3/3/3/: /9/4/4/4/4/1 /: /9/1/6/7 /3/7/5 /+ /2/6/4 /0/2 /3/3 /2 /3/7/5 /= /2/6/4 /: /9/7/2/2/6/: /9/4/4/4/4/1 /: /9/1/6/7 /3/7/5 /;;x /(/5/)/= /2/6/4 /0 /; /1 /1/1 /3 /0 /0/1 /2 /0 /0 /3/7/5 /2/6/4 /: /9/7/2/2/6/: /9/4/4/4/4/1 /: /9/1/6/7 /3/7/5 /+ /2/6/4 /0/2 /3/3 /2 /3/7/5 /= /2/6/4 /: /9/7/2/2/6/: /9/9/0/7/5/1 /: /9/8/6/1 /3/7/5and so on /(the exact solution of this equation is x /= h/1 /1 /2 iT/)/.Problem /7/./2/./1/. /Solv e the system giv en in example /7/./2/./1 b y the Gauss/-Seidel metho d/./2/./7/./3 Decomp osition of the System Matrix and Con v ergence ofthe Iterativ e pro cessBoth Jacobi/'s and the Gauss/-Seidel algorithms are of the t yp eM x /( k /+/1/)/= N x /( k /)/+ b/;; /(/6/)where A /= M /; N/: Assume that the de c omp osition of the matrix A is giv en/.Applying the iterativ e algorithm/, it is imp ortan t that the linear system /(/6/)with the system matrix M is easy to solv e/. By Jacobi/'s metho d M is adiagonal matrix and b y the Gauss/-Seidel metho d it is a lo w er triangularmatrix/. It app ears that the con v ergence of the iterativ e pro cess giv en b y /(/6/)dep ends on the sp ectral radius of the matrix M /; /1N /./1/6/9 De/nition /7/./3/./1/. The quan tit y/ /( G /) /= max fj / j /: / /2 / /( G /) gis the sp e ctr al r adius of the matrix G /2 C n / n/.Prop osition /7/./3/./1/. Let A /= M /; N b e the decomp osition of the regularmatrix A /2 R n / nand let b /2 R n/: If the matrix M is regular and/ /( M /; /1N /) /< /1 /;; /(/7/)then the sequence of the appro ximation f x /( k /)g de/ned b y algorithm /(/6/) con/-v erges to the solution x /= A /; /1b of system /(/1/) b y arbitrary initial appro xima/-tion x /(/0 /)/.Pr o of/. Let us gete /( k /)/= x /( k /)/; x /: /(/8/)Since the exact solution satis/es the equalit yM x /= N x /+ b /;; /(/9/)then from equalities /(/6/) and /(/9/) w e getM /( x /( k /+/1/)/; x /)/= N /( x /( k /)/; x /) /:T aking in to accoun t /(/8/)/, w e /nd for arbitrary non/-negativ e in teger k therelationM e /( k /+/1/)/= N e /( k /)ore /( k /+/1/)/= M /; /1N e /( k /)/=/( M /; /1N /) k /+/1e /(/0/)/:By virtue of prop osition /7/./1/./2/, it follo ws from inequalit y /(/7/) thatlimk /!/1 /( M /; /1N /) k/=/0 /:Th us/,limk /!/1 x /( k /)/= x /: /2Example /7/./3/./1/. /Let the matrix of the system A x /= b beA /= /2/6/4 /2 /0 /1/0 /1 /0/1 /0 /; /1 /3/7/5 /:/1/7/0 W e will pro v e that the sequence of the appro ximations f x /( k /)g de/ned b yJacobi/'s algorithm con v erges to the solution of the system for an y initialappro ximation x /(/0/)/.SinceA /= L /+ D /+ U /= /2/6/4 /0 /0 /0/0 /0 /0/1 /0 /0 /3/7/5 /+ /2/6/4 /2 /0 /0/0 /1 /0/0 /0 /; /1 /3/7/5 /+ /2/6/4 /0 /0 /1/0 /0 /0/0 /0 /0 /3/7/5 /;;Mj /= D /= /2/6/4 /2 /0 /0/0 /1 /0/0 /0 /; /1 /3/7/5 /;; Nj /= /; /( L /+ U /)/= /2/6/4 /0 /0 /; /1/0 /0 /0/; /1 /0 /0 /3/7/5andM /; /1j Nj /= /2/6/4 /1 /= /2 /0 /0/0 /1 /0/0 /0 /; /1 /3/7/5 /2/6/4 /0 /0 /; /1/0 /0 /0/; /1 /0 /0 /3/7/5 /= /2/6/4 /0 /0 /; /1 /2/0 /0 /0/1 /0 /0 /3/7/5 /;;then/ /( M /; /1j Nj /)/= //0 /;; /1 /2 i p /2 /;; /; /1 /2 i p /2 /and/ /( M /; /1j Nj /) /= max fj / j /: / /2 / /( M /; /1j Nj /) g /= /1 /2 p /2 /< /1 /:W e note that the matrix A is regular/. Hence/, b y virtue of prop osition /7/./3/./1/,the sequence of the appro ximations f x /( k /)g de/ned b y Jacobi/'s algorithm con/-v erges to the solution of the system x /= A /; /1b for an y initial appro ximationx /(/0/)/.Problem /7/./3/./1/. /Solv e the system/2/6/4 /; /2 /1 /; /1/; /1 /4 /; /2/; /1 /; /2 /; /2 /3/7/5 x /= /2/6/4 /3/2/; /4 /3/7/5b oth b y Jacobi/'s and the Gauss/-Seidel metho d/. Pro v e that the sequences ofthe appro ximations de/ned b y these algorithms con v erge to the solution ofthis system for an y initial appro ximation x /(/0/)/.Problem /7/./3/./2/. /Solv e the system/2/6/4 /2 /1 /; /1/; /1 /3 /; /1/; /1 /; /1 /4 /3/7/5 x /= /2/6/4 /5/3/0 /3/7/5/1/7/1 b oth b y Jacobi/'s and the Gauss/-Seidel metho d/. Pro v e that the sequences ofthe appro ximations de/ned b y these algorithms con v erge to the solution ofthis system for arbitrary initial appro ximation x /(/0/)/.De/nition /7/./3/./2/. The matrix A /2 C n / nis a matrix with the strictlydominant diagonal ifj aii j /> nXj /=/1j /6/= i j aij j /( i /=/1 /: n /) /:Remark /7/./3/./1/. If the matrix A /2 R n / nis a matrix with the strictlydominan t diagonal/, then the sp ectral radius / /( M /; /1J NJ /) of the matrix M /; /1J NJsatis/es the condition/ /( M /; /1J NJ /) /< /1 /;;i/.e/./, the iteration giv en b y form ula /(/4/) con v erges/.Pr o of/. See Golub/, Loan /(/1/9/9/6/, pp/. /1/2/0/, /5/1/2/)/. /2Prop osition /7/./3/./2/. If A /2 R n / nis a symmetric p ositiv e de/nite matrix/,then the Gauss/-Seidel iterativ e pro cess con v erges for arbitrary x /(/0/)/.Pr o of/. Let us denote A /= L /+ D /+ L T/;; where L is a strictly lo w ertriangular matrix /(zeros on the leading diagonal/) and D is a diagonal matrix/.Since the matrix A is p ositiv e de/nite/, then/, b y virtue of corollary /6/./2/./1/, thematrix D is also p ositiv e de/nite/. Hence p D exists/. The matrices A andL /+ D are regular/. Therefore/, b y virtue of prop osition /7/./3/./1/, to pro v e thecon v ergence of the Gauss/-Seidel iterativ e pro cess/, it is su/cien t to sho wthat the sp ectral radius / /( G /) of the matrix G /= /; /( L /+ D /) /; /1U satis/es thecondition / /( G /) /< /1 /: Let G/1 /= D /1 /= /2GD /; /1 /= /2/: Since the similar matrices Gand G/1 ha v e the same sp ectrum/, then it is su/cien t to c hec k the condition/ /( G/1 /) /< /1/. Let L/1 /= D /; /1 /= /2LD /; /1 /= /2/: W e /nd thatG/1 /= D /1 /= /2GD /; /1 /= /2/= /; D /1 /= /2/( L /+ D /) /; /1L TD /; /1 /= /2/=/= /; D /1 /= /2/( D /1 /= /2L/1 D /1 /= /2/+ D /1 /= /2ID /1 /= /2/) /; /1D /1 /= /2L T/1 D /1 /= /2D /; /1 /= /2/=/= /; D /1 /= /2/[ D /1 /= /2/( L/1 /+ I /) D /1 /= /2/] /; /1D /1 /= /2L T/1 D /1 /= /2D /; /1 /= /2/=/= /; D /1 /= /2D /; /1 /= /2/( L/1 /+ I /) /; /1D /; /1 /= /2D /1 /= /2L T/1 D /1 /= /2D /; /1 /= /2/= /; /( L/1 /+ I /) /; /1L T/1 /:Hence it is su/cien t to pro v e that / /( G/1 /) /< /1 /: If G/1 x /= / x /;; while x Hx /= /1/,then/; /( I /+ L/1 /) /; /1L T/1 x /= / x /;;/1/7/2 /; L T/1 x /= / /( I /+ L/1 /) xand/; x HL T/1 x /= / x Hx /+ / x HL/1 xand/; x HL T/1 x /= / /+ / x HL/1 xIf w e set x HL/1 x /= / /+ i/ /;; then x HL T/1 x /= / /; i/ and/; / /+ i/ / /= / /(/1 /+ / /+ i/ /) /;;and also/ /= /; / /+ i/ /1/+ / /+ i/ /:Th us/,j / j /2/= / /2/+ / /2 / /2/+/2 / /+/1/+ / /2 /: /(/1/0/)Since/, the matrix A is p ositiv e de/nite/, then/, b y virtue of prop osition /6/./2/./1/,the matrix D /; /1 /= /2AD /; /1 /= /2is also p ositiv e de/nite andD /; /1 /= /2AD /; /1 /= /2/= D /; /1 /= /2/( D /+ L /+ L T/) D /; /1 /= /2/=/= D /; /1 /= /2/( D /1 /= /2ID /1 /= /2/+ D /1 /= /2L/1 D /1 /= /2/+ D /1 /= /2L T/1 D /1 /= /2/) D /; /1 /= /2/=/= I /+ L/1 /+ L T/1 /:Hence/0 /< x H/( I /+ L/1 /+ L T/1 /) x /=/1 /+ x HL/1 x /+ x HL T/1 x /= /1/+/2 //=/1 /+ / /+ i/ /+ / /; i/ /=/2 / /+/1 /;;and condition /(/1/0/) implies j / j /< /1 /;; i/.e/./, / /( G/1 /) /< /1 /: /2/1/7/3 /2/./7/./4 Acceleration of the Con v ergence of an Iterativ e Pro cessIf the quan tit y / /( M /; /1G NG /) is smaller than one but close to one/, then theGauss/-Seidel metho d con v erges/, but v ery slo wly /. A problem arises ho wd o w eaccelerate the con v ergence of the sequence of appro ximations f x /( k /)g /? Oneof the pro cesses of acceleration of the con v ergence is the so/-called metho d ofr elaxation /. The relaxation metho d is based on the algorithm/ /( k /+/1/)i /= /! /( /i /; i /; /1Xj /=/1 aij / /( k /+/1/)j /; nXj /= i /+/1 aij / /( k /)j /) /=aii /+/( /1 /; /! /) / /( k /)i /( i /= /1/: n /) /;;whic h can be written in the matrix formM/! x /( k /+/1/)/= N/! x /( k /)/+ /! b /;;where M/! /= D /+ /!L and N/! /=/( /1 /; /! /) D /; /!U /: The problem lies in /ndingthe parameter /! so that / /( M /; /1/! N/! /) w ere the least/. By certain additionalconditions/, this problem can be solv ed/.The second w a y of acceleration of the con v ergence is the so/-called Cheby/-shev metho d of c onver genc ea c c eler ation/. Let us supp ose that w eh a v e foundusing algorithm /(/6/)/, the appro ximations x /(/1/)/;;/:/:/: /;; x /( k /)of the solution x ofsystem /(/1/)/. Lety /( k /)/= kXj /=/0 /j /( k /) x /( j /)/: /(/1/1/)The problem lies in /nding the co e/cien ts /j /( k /) in form ula /(/1/1/) so that theerror v ector y /( k /)/; x of the appro ximation y /( k /)is smaller than the error v ectorx /( k /)/; x /: In case ofx /( j /)/= x /( j /= /1 /: k /) /;;it is natural to demand that y /( k /)/= x /: This is just ac a s e whenkXj /=/0 /j /( k /)/= /1 /: /(/1/2/)Ho w do w e c ho ose the factors /j /( k /)s o that they w ould satisfy /(/1/2/)/, and theerror v ector w ere the shortest/? Sincex /( k /)/; x /=/( M /; /1N /) ke /(/0/)/;;/1/7/4 theny /( k /)/; x /= kXj /=/0 /j /( k /)/( x /( j /)/; x /)/= kXj /=/0 /j /( k /)/( M /; /1N /) je /(/0 /)/=/= kXj /=/0 /j /( k /) G je /(/0 /)/= pk /( G /) e /(/0/)/;;where G /= M /; /1N andpk /( z /)/= kXj /=/0 /j /( k /) z j/:F rom condition /(/1/2/)/, it follo ws that pk /(/1/) /= /1 /: In addition/,/ / / y /( k /)/; x / / / /2 /k pk /( G /) k/2 / / / e /(/0/) / / / /2 /: /(/1/3/)W e con/ne ourselv es further to the case of a symmetric matrix G /. Let theeigen v alues /i of the symmetric matrix G satisfy the c hain of inequalities/; /1 /</ / /n / /:/:/: / //1 / / /< /1 /:If / is the eigen v alue corresp onding to the eigen v ector x /, thenpk /( G /) x /= kXj /=/0 /j /( k /) G jx /= kXj /=/0 /j /( k /) / jx /;;i/.e/./, the v ector x is also the eigen v ector of the matrix pk /( G /) corresp ondingto the eigen v aluekXj /=/0 /j /( k /) / j/= pk /( / /) /:In the case of the symmetric matrix G /, the matrix pk /( G /) is also symmetric/.Hencek pk /( G /) k/2 /= max/i /2 / /( G /) j kXj /=/0 /j /( k /) / ji j/ max/ /2 /[ //;;/ /] j pk /( / /) j /:T o decrease the quan tit y k pk /( G /) k/2 one m ust /nd a p olynomial pk /( z /) that hassmall v alues on the segmen t/[ //;; / /] and satis/es the condition pk /(/1/) /= /1 /: TheChebyshev p olynomials ha v e these prop erties/. The Chebyshev p olynomialsare de/ned on the segmen t /[ /; /1/;; /1/] b y the recurrence relationcj /( z /)/=/2 zcj /; /1 /( z /) /; cj /; /2 /( z /) /(j /= /2/: /)/,/1/7/5 while c/0 /( z /) /= /1 and c/1 /( z /)/= z/: These p olynomials satisfy the inequalit yj cj /( z /) j/ /1 z /2 /[ /; /1/;; /1/]and cj /(/1/) /= /1/, and the v alues j cj /( z /) j gro w quic kly outside the segmen t/[ /; /1/;; /1/] /:The p olynomialpk /( z /)/= ck /( /; /1/+ /2 /( z /; / /) /= /( / /; / /)/) ck /( / /) /;; /(/1/4/)where/ /= /; /1/+ /2 /1 /; / / /; / /= /1/+/2 /1 /; / / /; / /> /1 /;;satis/es the conditions pk /(/1/) /= /1 andj pk /( z /) j/ /1 z /2 /[ / /;; / /] /:T aking in to accoun t relations /(/1/3/) and /(/1/4/)/, w e /nd/ / / y /( k /)/; x / / / /2 / / / / x /( k /)/; x / / / /2 j ck /( / /) j /:Therefore/, the greater is //;; the greater is j ck /( / /) j /, and the greater will b e theChebyshev/'i ac c eler ation/./2/./8 Numeric Stabilit yIn the follo wing part w e will consider ho w the deviations of the en tries ofthe regular matrix A /2 R n / nand v ector b /2 R nwill cause the deviations ofthe solution x of the linear systemA x /= b /: /(/1/)/1/7/6 /2/./8/./1 Singular V alue Decomp osition and Numeric Stabilit yDe/nition /8/./1/./1/. The /" /; r ank of the matrix A is de/ned b y the form ular ank /( A/;; /" /)/= mink A /; B k/ /" r ank /( B /) /:Example /8/./1/./1/. /Let us /nd the /" /-rank of the matrixA /= /"/0 /: /1 /0 /: /0/1/; /0 /: /1 /0 /: /0/1 /#if /" /=/0 /: /1/4 /:Eviden tly /,/0 / r ank /( A/;; /" /) / /2 /:If the equalit yr ank /( A/;; /" /)/= /0 /;;holds/, then/9 B /: r ank /( B /)/= /0 /^ k A /; B k/2 / /"/:Sincera n k /( B /)/= /0 /, B /= /"/0 /0/0 /0 /#/;;thenk A /; B k/2 /= / / / / / /"/0 /: /1 /0 /: /0/1/; /0 /: /1 /0 /: /0/1 /#/; /"/0 /0/0 /0 /# / / / / / /2 /=/= / / / / / /"/0 /: /1 /0 /: /0/1/; /0 /: /1 /0 /: /0/1 /# / / / / / /2 /= max /q //1 /;; q //2 //;;where //1 and //2 are the eigen v alues of the matrix/"/0 /: /1 /; /0 /: /1/0 /: /0/1 /0 /: /0/1 /#/"/0 /: /1 /0 /: /0/1/; /0 /: /1 /0 /: /0/1 /#/= /"/0 /: /0/2 /0/0 /0 /: /0/0/0/2 /#/:Therefore/, //1 /=/0 /: /0/2 and //2 /=/0 /: /0/0/0/2 andk A /; B k/2 /= p /0 /: /0/2 / /0 /: /1/4/1/4/2 /> /0 /: /1/4 /= /"/:/1/7/7 This is a con tradiction/, and it means that r ank /( A/;; /" /) /> /0 /: Let us sho w thatra n k /( A/;; /" /)/= /1 /: No w/, for the matrixB /= /"/0 /: /1 /0 /: /0/1/0 /0 /#ra n k /( B /)/= /1 a n dk A /; B k/2 /= / / / / / /"/0 /: /1 /0 /: /0/1/; /0 /: /1 /0 /: /0/1 /#/; /"/0 /: /1 /0 /: /0/1/0 /0 /# / / / / / /2 /=/= / / / / / /"/0 /0/; /0 /: /1 /0 /: /0/1 /# / / / / / /2 /= max /q //1 /;; q //2 //;;where //1 and //2 are the eigen v alues of the matrix/"/0 /; /0 /: /1/0 /0 /: /0/1 /#/"/0 /0/; /0 /: /1 /0 /: /0/1 /#/= /"/: /0/1 /; /: /0/0/1/; /: /0/0/1 /: /0/0/0/1 /#/:Th us/, //1 /=/0 /: /0/1 and //2 /=/0 /: /0/0/0/1/, andk A /; B k/2 /= p /0 /: /0/1 / /0 /: /1 /< /0 /: /1/4 /= /"/:But this means that ra n k /( A/;; /" /)/=/1 /:Prop osition /8/./1/./1/. Let A /= U / V Tb e the singular v alue decomp ositionof a matrix A /2 R m / n/. If k/< r /= r ank /( A /) andAk /= kXi /=/1 /i ui v Ti /;;thenminra n k /( B /)/= k k A /; B k/2 /= k A /; Ak k/2 /= /k /+/1 /:Pr o of/. SinceU TAk V /= U T kXi /=/1 /i ui v Ti V /=/= hu T/1 /
/
/
u Tm iT kXi /=/1 /i ui v Ti hv/1 /
/
/
vn i/=/= hu T/1 /
/
/
u Tm iT hPki /=/1 /i ui v Ti v/1 /
/
/
Pki /=/1 /i ui v Ti vn i/=/1/7/8 /= /2/6/6/4 u T/1/././.u Tm /3/7/7/5 h//1 u/1 /
/
/
/k uk /0 /
/
/
/0 i/=/= /2/6/6/4 //1 u T/1 u/1 /
/
/
/k u T/1 uk /0 /
/
/
/0/././. /././. /././. /./././/1 u Tm u/1 /
/
/
//1 u Tm uk /0 /
/
/
/0 /3/7/7/5 /=/= diag /( //1 /;;/:/:/: /;;/k /;; /0 /;;/:/:/: /;; /0/) /2 R m / nandU T/( A /; Ak /) V /= diag /(/0 /;;/:/:/: /;; /0 /;;/k /+/1 /;;/:/:/: /;;/p /) /;;while p /= min f m/;; n g /: Since the Euclid norm of the matrix A /; Ak equals thegreatest en try of the matrix U T/( A /; Ak /) V /,t h e nk A /; Ak k/2 /= /k /+/1 /:Let B /2 R m / nbe a matrix for whic h r ank /( B /)/= k/: W e can /nd the ortho/-normal v ectors x/1 /;;/:/:/: /;; xn /; k /;; suc h that the n ull space of the matrix B is alinear span of the v ectors x/1 /;;/:/:/: /;; xn /; k /, i/.e/./,N /( B /)/= span f x/1 /;;/:/:/: /;; xn /; k g /:Since in the space R nn /+/1 v ectors are linearly dep enden t/, thenspan f x/1 /;;/:/:/: /;; xn /; k g/\ span f v/1 /;;/:/:/: /;; vk /+/1 g /6/= f /0 g /:If z is a unit v ector /(b y the Euclidean norm/) from this in tersection/, thenB z /= /0 andA z /= rXj /=/1 /j uj v Tj k /+/1Xi /=/1 /( v Ti z /) vi /= rXj /=/1 /j uj k /+/1Xi /=/1 /( v Ti z /)/( v Tj vi /)/=/= rXj /=/1 /j uj k /+/1Xi /=/1 /( v Ti z /) /ij /= k /+/1Xi /=/1 /i /( v Ti z /) ui /:Hencek A /; B k /2/2 /k /( A /; B /) z k /2/2 /= k A z k /2/2 /= k /+/1Xi /=/1 / /2i /( v Ti z /) /2//1/7/9 / k /+/1Xi /=/1 / /2k /+/1 /( v Ti z /) /2/= / /2k /+/1 k /+/1Xi /=/1 /( v Ti z /) /2/= / /2k /+/1 /: /2Corollary /8/./1/./1/. If the matrix A /2 R n / nis regular/, then the leastsingular v alue /n of the matrix A determines the distance of the matrix Afrom the nearest singular matrix/.Corollary /8/./1/./2/. If r/" /= r ank /( A/;; /" /) /;; then//1 / /:/:/: / /r/" />/" / /r/" /+/1 / /:/:/: / /p /( p /= min f m/;; n g /) /:Problem /8/./1/./1/. /Use corollary /8/./1/./2 to solv e the problem giv en in exam/-ple /8/./1/./1/.Prop osition /8/./1/./2/. IfA /= nXi /=/1 /i ui v Ti /= U / V Tis the singular v alue decomp osition of the regular matrix A /2 R n / n/, thenthe solution x of system /(/1/) can be expressed in the formx /= A /; /1b /=/( U / V T/) /; /1b /= nXi /=/1 u Ti b /i vi /: /(/2/)Pr o of/. Let us c hec k the correctness of the assertion of the prop osition/:x /= A /; /1b /=/( U / V T/) /; /1b /= V / /+U Tb /=/= nXi /=/1 vi u Ti b /=/i /= nXi /=/1 vi /( u Ti b /) /=/i /= nXi /=/1 u Ti b /i vi /:Corollary /8/./1/./3/. F rom the represen tation of the solution in form /(/2/)it app ears that small deviations of the en tries of the matrix A could causegreat deviations of the solution x if /n is small/.Example /8/./1/./2/. /F or whic h system/"/5/4 /= /1/2/5 /; /1/6/9 /= /2/5/0/5/3 /= /1/2/5 /; /5/4 /= /1/2/5 /#/"//1//2 /#/= /"/2/1 /#/1/8/0 or/"/4/1 /= /1/6/2/5/0 /; /9/9 /= /1/3/0/0/0/4/6 /= /1/5/1/0/1 /; /1/7 /= /3/2/5/0 /#/"//1//2 /#/= /"/2/1 /#small deviations of the en tries of the system matrix can cause greater devia/-tions of the solution x /? Applying the pac k age /\Maple/"/, w e /nd the singularv alue decomp osition of b oth system matrices/:/"/5/4 /= /1/2/5 /; /1/6/9 /= /2/5/0/5/3 /= /1/2/5 /; /5/4 /= /1/2/5 /#/= /"/; /: /8 /; /: /6/; /: /6 /: /8 /#/"/1 /: /0 /0/0 /: /1 /#/"/; /: /6 /: /8/: /8 /: /6 /#and/"/4/1 /= /1/6/2/5/0 /; /9/9 /= /1/3/0/0/0/4/6 /= /1/5/1/0/1 /; /1/7 /= /3/2/5/0 /#/= /"/; /: /8 /; /: /6/; /: /6 /: /8 /#/"/: /0/1 /0/0 /: /0/0/1 /#/"/; /: /3/8/4/6/2 /: /9/2/3/0/8/: /9/2/3/0/8 /: /3/8/4/6/2 /#/:W e see that the least singular v alue //2 of the /rst system matrix is h undredtimes smaller than the least singular v alue of the second system matrix/.Therefore/, in virtue of corollary /8/./1/./3/, w e can state that the /rst system ismore stable than the second one/, i/.e/./, small deviations of the en tries of thesecond system matrix can cause greater deviations of the solution x than thedeviation of the same order of the en tries of the /rst system matrix/.Problem /8/./1/./2/. /Whic h of the t w o systems/"/5/4/0/0 /= /1/6/9 /; /3/9/4/0 /= /1/6/9/1/4/6/5/0 /= /1/6/9 /; /5/4/0/0 /= /1/6/9 /#/"//1//2 /#/= /"/1/1/7 /#or/"/1/4/1 /= /8/4/5 /; /8/4/1 /= /8/5/0/2/9/1 /= /6/7/6 /; /1/4/1 /= /8/4/5 /#/"//1//2 /#/= /"/1/1/7 /#is more stable/?/2/./8/./2 T a ylor Dev elopmen tW e can obtain the exact ev aluation of the sensibilit y of system /(/1/) usingthe system dep ending on a parameter/( A /+ /F /) x /( / /)/= b /+ / f /;; /(/3/)/1/8/1 where F /2 R n / n/, f /2 R nand x /(/0/) /= x /: If A is a regular matrix/, then x /( / /)is a di/eren tiable function of the parameter / in some neigh bo u r h ood o f t h ev alue /0/. Di/eren tiating the bo t h sides of equalit y /(/3/) with resp ect to theparameter / /,w e getF x /( / /)/+/( A /+ /F /) d x d/" /( / /)/= fandd x d/ /( / /)/= /( A /+ /F /) /; /1/( f /; F x /( / /)/) /(/4/)It follo ws from relation /(/4/) thatd x d/ /(/0/) /= A /; /1/( f /; F x /) /:Let us write the /rst order T a ylor form ula for the function x /( / /)x /( / /)/= x /+ / d x d/ /(/0/) /+ O /( / /2/)/= x /+ /A /; /1/( f /; F x /)/+ O /( / /2/) /:As the result/, w e obtain for arbitrary v ector norm and for the matrix normcorresp onding to itk x /( / /) /; x k k x k /= / / / /( / d x d/ /0/) /+ O /( / /2/) / / / k x k /= k /A /; /1/( f /; F x /)/+ O /( / /2/) k k x k //j / j k A /; /1k /( k f k /+ k F k/
k x k /) k x k /+ O /( / /2/) /j / j / / / A /; /1 / / / /( k f k k x k /+ k F k /)/+ O /( / /2/) /:T aking in to accoun t that from relation /(/1/) it follo ws k b k / k A kk x k /;; w eobtain the estimatek x /( / /) /; x k k x k /j / j / / / A /; /1 / / / k A k /( k f k k b k /+ k F k k A k /)/+ O /( / /2/) /;;ork x /( / /) /; x k k x k / k /( A /)/( /"re l /( A /)/+ /"re l /( b /)/) /+ O /( / /2/) /;;where /"re l /( A /) /= j / j k F k k A k and /"re l /( b /) /= j / j k f k k b k are the relativ e errors of thematrix A and the v ector b /, resp ectiv ely /./1/8/2 Prop osition /8/./2/./1/. If A /2 R n / nis a regular matrix/, then the relativ eerror /"re l /( x /) of the solution of the linear system /(/1/) corresp onding to therelativ e error /"re l /( A /) of the matrix A and the relativ e error /"re l /( b /) /;; of thev ector b is giv en b y/"re l /( x /) / k /( A /)/( /"re l /( A /)/+ /"re l /( b /)/) /:Corollary /8/./2/./1/. In case of the Euclidean norm the estimate the/"re l /( x /) / //1 /n /( /"re l /( A /)/+ /"re l /( b /)/) /:holds/.Pr o of/. The relation k A k/2 /= //1 holds/. As the matrix A is regular/, it follo wsfrom its singular v alue decomp osition A /= U / V Tthat A /; /1/= V / /+U T/;; where/ /+/= diag /(/1 /=//1 /;;/:/:/: /;; /1 /=/n /) /: Since max/1 / i / n /1 /=/i /= /1 /=/n /;; then k A /; /1k/2 /= /1 /=/nandk/2 /( A /)/= k A k/2 k A /; /1k/2 /= //1 /=/n /: /2Example /8/./2/./1/. /Let us estimate the relativ e error /"re l /( x /) of the solutionx of the system A x /= b in the case of the Euclidean norm ifA /= /2/6/4 /8/0 /1/8 /2/4/6/0 /; /2/4 /; /3/2/0 /4 /= /5 /; /3 /= /5 /3/7/5 /;;/"re l /( A /)/= /0 /: /0/9 and /"re l /( b /)/= /0 /: /0/1 /:Let us /nd the singular decomp osition of the matrix AA /= /2/6/4 /8/0 /1/8 /2/4/6/0 /; /2/4 /; /3/2/0 /4 /= /5 /; /3 /= /5 /3/7/5 /=/= /2/6/4 /; /: /8 /: /6 /0/; /: /6 /; /: /8 /0/0 /0 /1 /: /0 /3/7/5 /2/6/4 /1/0/0 /: /0 /0 /0/0 /5/0 /: /0 /0/0 /0 /1 /: /0 /3/7/5 /2/6/4 /; /1 /: /0 /0 /0/0 /: /6 /: /8/0 /: /8 /; /: /6 /3/7/5 /:In virtue of corollary /8/./2/./1/, w e can state/"re l /( x /) / //1 //3 /( /"re l /( A /)/+ /"re l /( b /)/) /= /1/0/0 /1 /(/0 /: /0 /9/+/0 /: /0/1/) /= /1/0 /:/1/8/3 Problem /8/./2/./1/. /Estimate the relativ e error /"re l /( x /) of the solution x ofthe system A x /= b in case of the Euclidean norm ifA /= /2/6/4 /6/5 /; /1/4/4 /= /1/3 /6/0 /= /1/3/1/5/6 /6/0 /= /1/3 /; /2/5 /= /1/3/0 /5 /= /1/3 /1/2 /= /1/3 /3/7/5 /;;/"re l /( A /)/= /0 /: /0/0/8 and /"re l /( b /)/= /0 /: /0/0/2 /:Remark /8/./2/./1/. Kahan /(/1/9/6/6/) pro v ed that/1 kp /( A /) /= minA /+/
A singular k /
A kp k A kpand Rice /(/1/9/6/6/) pro v ed thatk /( A /) /= lim/" /! /0 supk /
A k/ /" k A k k /( A /+/
A /) /; /1/; A /; /1k /" /
/1 k A /; /1k /:/2/./8/./3 Strict EstimationsThe assertion of prop osition /8/./2/./1 is of a lo cal kind/. Namely /, this prop osi/-tion is pro v ed for relativ ely small deviations/. Next w e will giv et h e e v aluationof the deviation of the solution in general case/. First w e will pro v e one nec/-essary auxiliary result/.Prop osition /8/./3/./1/. If the matrix A /2 R n / nis a regular matrix andr / / / / A /; /1E / / / p /< /1 /;;then A /+ E is regular and/ / / /( A /+ E /) /; /1/; A /; /1 / / / p / k E kp k A /; /1k /2p /1 /; r /:Pr o of /. A regular matrix can b e represen ted in the formA /+ E /= A /( I /; F /) /;;/1/8/4 where F /= /; A /; /1E/: Since k F kp /= r/< /1 /;; then/, in virtue of prop osition /7/./1/./1/,the matrix I /; F is regular and/ / / /( I /; F /) /; /1 / / / p /< /1 /1 /; r /:Hence/( A /+ E /) /; /1/=/( I /; F /) /; /1A /; /1and/ / / /( A /+ E /) /; /1 / / / p / jj A /; /1jjp /1 /; r /:F rom the equalit yB /; /1/= A /; /1/; B /; /1/( B /; A /) A /; /1it follo ws that/( A /+ E /) /; /1/; A /; /1/= /; A /; /1E /( A /+ E /) /; /1and/ / / /( A /+ E /) /; /1/; A /; /1 / / / p / / / / A /; /1 / / / p k E kp / / / /( A /+ E /) /; /1 / / / p / k A /; /1k /2p k E kp /1 /; r /: /2Prop osition /8/./3/./2/. LetA x /= b /;; A /2 R n / n/;; /0 /6/= b /2 R n/;;/( A /+/
A /) y /= b /+/
b /;; /
A /2 R n / n/;; /
b /2 R n/;;while k /
A k/ / k A k and k /
b k/ / k b k /: If / /
k /( A /) /= r /< /1 /;; then A /+/
Ais a regular matrix/, andk y k k x k / /1/+ r /1 /; randk y /; x k k x k / /2 / /1 /; r k /( A /) /:Pr o of/. Since k A /; /1/
A k/ / k A /; /1kk A k /= r/< /1 /;; then/, in virtue of prop osi/-tion /8/./3/./1/, A /+/
A is a regular matrix/. Applying prop osition /7/./1/./1 and theequalit y/( I /+ A /; /1/
A /) y /= x /+ A /; /1/
b /;;w e /ndk y k/ / / / /( I /+ A /; /1/
A /) /; /1 / / / /( k x k /+ / / / / A /; /1 / / / k b k /) //1/8/5 / /1 /1 /; r /( k x k /+ / / / / A /; /1 / / / k b k /)/= /1 /1 /; r / k x k /+ r k b k k A k /!/:In addition/, w e /nd that k b k /= k A x k/k A kk x k /: Hencek y k/ /1 /1 /; r /( k x k /+ r k x k /) /;;and that means that the /rst part of the assertion is true/. Since the relationy /; x /= A /; /1/
b /; A /; /1/
y /;;holds/, thenk y /; x k/ / / / / A /; /1 / / / k b k /+ / / / / A /; /1 / / / k A kk y kand therefore/,k y /; x k k x k / /k /( A /) k b k k A kk x k /+ /k /( A /) k y k k x k // /k /( A /) //1/+ /1/+ r /1 /; r //= /2 / /1 /; r k /( A /) /: /2Example /8/./3/./1/. /LetA /= /"/5/2 /: /8 /6/0 /: /4/2/9 /: /6 /5/2 /: /8 /#/^ k /
A k/2 / /8 /^ b /= /"/4/0/3/0 /#/^ k /
b k/2 / /4 /:W e will estimate the relativ e error of the solution of the system A x /= b usingthe Euclidean norm/. SinceA TA /= /"/5/2 /: /8 /2/9 /: /6/6/0 /: /4 /5/2 /: /8 /#/"/5/2 /: /8 /6/0 /: /4/2/9 /: /6 /5/2 /: /8 /#/= /"/3/6/6/4 /4/7/5/2/4/7/5/2 /6/4/3/6 /#and the eigen v alues A TA are //1 /= /1/0/0/0/0 and //2 /= /1/0/0 /;; thenk A k/2 /= max /q //1 /;; q //2 //=/1 /0 /0 /:Let us /nd the Euclidean norm of the v ector b /:k b k/2 /= p /4/0 /2/+/3 /0 /2/=/5 /0 /:/1/8/6 If one tak es / /=/0 /: /0/8 /;; then the conditions k /
A k/ / k A k and k /
b k/ / k b kare satis/ed/. F rom the singular v alue decomp osition of the matrix A/"/5/2 /: /8 /6/0 /: /4/2/9 /: /6 /5/2 /: /8 /#/= /"/; /: /8 /; /: /6/; /: /6 /: /8 /#/"/1/0/0 /: /0 /0/0 /1/0 /: /0 /#/"/; /: /6 /; /: /8/; /: /8 /: /6 /#w e /nd thatk/2 /( A /)/= //1 //2 /= /1/0/0 /1/0 /=/1 /0 /:Hencer /= / /
k/2 /( A /)/= /0 /: /0/8 /
/1/0 /= /0 /: /8 /< /1 /;;and/, in virtue of prop osition /8/./3/./2/, w e getk /
x k/2 k x k/2 / /2 / /1 /; r k/2 /( A /)/= /2 /
/0 /: /8 /1 /; /0 /: /8 /
/1/0 /= /8 /:Problem /8/./3/./1/. /LetA /= /"/6/5 /; /1/2/; /1/5/6 /; /5 /#/^ k /
A k/2 / /1/6/9 /= /1/5 /^ b /= /"/1/2/0/5/0 /#/^ k /
b k/2 / /1/3 /= /1/5 /:Find the relativ e error of the solution of the system A x /= b /. Use the Euclid/-ean norm/./1/8/7 ReferencesG/.H/.Golub and C/.F/.V an Loan /(/1/9/9/6/)/. Matrix Computations/, John Hop/-kinsUniv ersit y Press/, London/.G/.Kangro /(/1/9/6/2/)/. Higher Algebra/, Estonian State Publishers/, T allinn/(in Estonian/)/.P /.Lank aster /(/1/9/8/2/)/. Theory of Matrices/, Nauk a/, Mosco w /(in Russian/)/.E/.Oja and P /.Oja /(/1/9/9/1/)/. F unctional Analysis/, T artu Univ ersit y Press/,T artu /(in Estonian/)/.G/.Strang /(/1/9/8/8/)/. Linear Algebra and its Applications/, Harcourt BraceJo v ano vic h College Publishers/, Orlando/, Florida/./1/8/8