All Appendices

This was installed 10.22.16, do not edit here! Appendix A: Matrix rank equals the number of independent columns and rows 1 Appendix B: Determinants 5 B.1 Definition of the determinant 5 B.2 The permutation group, the permutation tensor ε, and expansions for det(M) 8 B.3 Minors, Cofactors, and the Cofactor Expansions of det(M) 14 B.4 Expressions for the inverse matrix M-1 and Cramer's Rule 16 Appendix C: The Intersection of Constraint Surfaces 19 Appendix D: Lagrange Multipliers in Classical Mechanics 24 References 38 Appendix A: Matrix rank equals the number of independent columns and rows This Appendix proves our claim of (6.1.7) that, for a general m x n matrix M, the number of linearly independent columns and the number of linearly independent rows are the same, and both numbers are equal to the rank of the matrix. The reader will appreciate that this fact is non-obvious. Shilov proves it but the proof is a bit spread out over several sections. Here we provide a self-contained alternate proof. We first quote a series of square-matrix theorems that are proved from scratch in our Appendix B. These are doubtless very familiar to the reader, but we just want to get them stated. The derivations of these theorems may be less familiar. Theorem 1: det(MT) = det(M). Switching rows with columns does not change a determinant. (B.1.10) Theorem 2: det(M) can be represented in these two ways, where ε is the permutation tensor (B.2.8) : det(M) = Σaa... a εaa... a M1aM2a ...... Mna (B.2.10a) det(M) = Σaa... a εaa... a Ma1Ma2 ...... Man . (B.2.10b) Theorem 3: For a square matrix M, adding a multiple of one row to another does not change det(M). The same is true for adding a multiple of one column to another. (B.2.12) Theorem 4: Swapping two rows (columns) of square matrix M causes det(M) → - det(M). (B.2.13) Corollary 4: If two rows (columns) of square matrix M are the same, then det(M) = 0. (B.2.14) Theorem 5: det(AB) = det(A)det(B) (B.2.15) Corollary 5: det(ABC) = det(A)det(B)det(C) and so on. (B.2.16) Theorem 6 (Cofactor Expansions): det(M) = Σn Msn cof(Msn) = Σn (rs)n cof(Msn) work across row s s = 1,2....n (B.3.16a) det(M) = Σn Mns cof(Mns) = Σn (cs)n cof(Mns) work down column s s = 1,2....n (B.3.16b) Theorem 7: M-1 = = = for square matrix M. (B.4.8) Theorem 8 (Cramer's Rule): y = Mx xs = (B.4.14) To this list, we shall now add four new Theorems which will be proven right here. The definition of linear independence and linear dependence is given in the discussion surrounding (6.1.6) and won't be repeated here. Theorem 9: Columns of square matrix M are linearly dependent det(M) = 0 (A.1) Contrapositives: Columns of square matrix M are linearly independent det(M) ≠ 0 If we can prove Theorem 9 as stated, then the theorem is also true for rows. The reason is that swapping rows and columns corresponds to M ↔ MT and Theorem 1 says det(MT) = det(M). Proof of : According to (6.1.6) and nearby discussion, our premise of linear dependence says that either one or more columns of M are zero, or at least one column can be written as a linear combination of the other columns, so perhaps cj = Σi≠jkici . If a column is zero, det(M) = 0 from Theorem 6 going down that column. If cj = Σi≠jkici, one can add -Σi≠jkici to cj without changing det(M) according to Theorem 3. But this makes the new column j vanish, so again det(M) = 0. Proof of : Here we shall prove the contrapositive instead of the claim: claim: det(M) = 0 the columns of M are linearly dependent contrapositive: the columns of M are linearly independent det(M) ≠ 0 If the column vectors ci of matrix M are linearly independent, then (6.1.6) says Σxici = 0 x = 0. Therefore we know that ΣxiMji = 0 xj = 0 which is the same as Mx = 0 x = 0. Thinking of M:En → En, we claim that mapping Mx = y is one-to-one. Certainly each x goes into a single y, Could a y map back into two different values of x, call them x and x' with x ≠ x' ? Then Mx = y and Mx' = y . Subtract to get M(x-x') = 0. But since Mz = 0 z = 0 (as just shown), we find x = x' . Thus, the mapping Mx = y really is one-to-one, and that means it is invertible and the inverse is unique. From Theorem 7 the inverse is in fact given by M-1 = cof(MT)/det(M). For M-1 to exist, one must have det(M) ≠ 0. Another proof is very simple but perhaps less convincing. One can show (see our Tensor Analysis document in Refs) that if a set of column vectors ci of matrix M spans an n-piped in En, the "volume" of that n-piped is given by V = |det(M)|. If those vectors are linearly independent, then V ≠ 0. This is obvious in 2D (volume = area) and 3D but less obvious for general En. For example, in 3D if a third vector lies in the plane of the other two (is dependent), there is no volume. Theorem 10 (mini-columns theorem). Let R be a subset of N rows of nxm matrix A, and let S be a subset of columns {ci} in A. The matrix elements of A included in the intersection of these two sets form a set of "mini-columns" {Ci}. This set of mini-columns forms a matrix B within A. Matrix B could be contiguous, or it could be non-contiguous in one or both directions. The claim and its contrapositive are: (a) {Ci} linearly independent in B {ci} linearly independent in A (b) {ci} linearly dependent in A {Ci} linearly dependent in B (A.2) Below we shall prove (b) which then implies (a). A picture is worth a thousand words. Here matrix B happens to be contiguous in both directions : (A.3) Proof of (b): If the set {ci} is linearly dependent in A, then from (6.1.6) one can write ΣiS kici = 0 with k ≠ 0. This is a set of m equations which contains as a subset the set of N equations ΣiS kiCi = 0. This last equation then says that the {Ci} are linearly dependent in B. There is of course a similar "mini rows theorem". Theorem 11. If all kxk minors in k columns of matrix A vanish, the k columns are linearly dependent. (A.4) Contrapositive: If k columns are linearly independent, they must contain at least one non-zero kxk minor. Once proven for columns, replacing A→AT gives the same theorem for rows. Proof: Assume matrix A has m rows and n columns. In order for kxk minors to exist, one must have k ≤ min(m,n). Gather up the k columns and make them be the leftmost k columns of a new matrix B. Since these columns have m elements, and since k ≤ m, add m-k arbitrary new columns to the right of the k columns so that matrix B is then a square matrix. On the left below we show the k columns taken from matrix A in red, and then the arbitrary added columns are shown in blue. (A.5) Consider now the process of computing det(B) by going down the rightmost column using the standard cofactor sum formula of Theorem 6. This det(B) is a linear combination of (m-1)x(m-1) minors all in the left m-1 columns. Consider one of these minors. If we evaluate it using the same cofactor formula (of one less dimension), it will be a linear combination of (m-2)x(m-2) minors in the left m-2 columns. We keep going until we are evaluating a set of kxk cofactors in the left k columns. But these all vanish by the theorem premise. Thus, reversing this logic we conclude that det(B) = 0. The picture on the right above shows one minor at each level of the descent just described (red dot goes with red minor, etc). Now suppose it were possible that the k red columns were independent. Since the added blue columns are arbitrary, we could certainly find a set of added blue columns so that all m columns were independent. But then we would have det(B) ≠ 0 by Theorem 9. But we just showed that det(B) = 0, so it must not be possible to have the k red columns be independent, so they must be dependent. We are finally ready for the Main Act. Recall first that : Definition: The rank r of an m x n matrix A is the dimension of the largest non-vanishing minor within A. [Shilov 1.92 ] Thus, r ≤ min(m,n). (6.1.5) (A.6) Theorem 12. For a general nxm matrix A, (A.7) rank(A) = r A has exactly r linearly independent columns Once this theorem is proved for columns, it is also true for rows since rank(AT) = rank(A) by (A.6). Proof of : If rank(A) = r, A must have at least one non-vanishing rxr minor. Think of this minor being a set of mini-columns as in Theorem 10. Since then det(minor) ≠ 0, this set of mini-columns is linearly independent according to Theorem 9. Thus, the set of corresponding full columns (those that pass down through the minor) are linearly independent from Theorem 10 (a). So matrix A has at least these r independent columns. Now let k = r+1. Since rank(A) = r we know that all kxk minors in A vanish. By Theorem 11, any set of k columns must be linearly dependent. Since k = r+1, any r+1 columns are linearly dependent, so the number of independent columns is just r, the number of columns passing down through the minor. Proof of : If there are r linearly independent columns, then at least one rxr minor within those columns must be non-zero (contrapositive of Theorem 11). Suppose there were an (r+k)x(r+k) non-vanishing minor. Then the r+k mini-columns of that minor would be independent, and thus by Theorem 10 there would be r+k independent full columns. But the theorem premise says there are only r independent columns, so all minors larger than rxr must vanish. Therefore rank(A) = r. Sometimes the number of linearly independent rows of a matrix is called the row rank, and the number of independent columns is called the column rank. Theorem 12 and its row version show that column rank = row rank = rank = dimension of largest non-vanishing minor (A.8) Basis Minors and Basis Columns. If rank(A) = r, we know there must exist at least one non-vanishing rxr minor in A. Shilov refers to such a minor as a basis minor and the columns passing through this minor are called basis columns. We showed in Theorem 10 that the set of such basis columns is linearly independent (which is why they are called basis columns). Obviously any of these columns can be written as a linear combination of the basis columns, such as c2 = Σi=1k kici = 1 c2. If k = r+1, all kxk minors vanish and by Theorem 11 all sets of k = r+1 columns are linearly dependent. Thus, every non-basis column is a linear combination of the basis columns. We have thus proved Shilov's Basis Minor Theorem which we quote from section 1.93 of his book (p 25) (A.9) In his proof that column rank = rank, Shilov uses the above theorem as a starting point. Appendix B: Determinants An alterative title for this Appendix might be "More than you really want to know about determinants". We find it convenient to gather these Facts all in one place. Many of these Facts are used in Appendix A where we prove other Facts about matrix rank which in turn are used in Section 6 to support our alternate derivation of the Method of Lagrange Multipliers. It is hoped that the encapsulated presentation below might be useful to the reader for other applications of determinants. Of special interest to us is that permutations form a group which implies certain "rearrangement theorems" which in turn provide quick proofs of many well-known and some lesser-known Facts about determinants. B.1 Definition of the determinant First, we define z0 to be the n-component vector of increasing integers 1 to n, z0 ≡ . (B.1.1) Let a be some permutation (reordering) of these integers, so write a ≡ = A = A z0 (B.1.2) where A is an nxn matrix which has 1's in the right places to create this permutation vector a. For example, a = Azo ↔ = . The vector a can be obtained from the vector z0 by making Sa pairwise swaps of elements of z0. Although Sa is not unique, the number (-1)S is unique. For example, to get from (1,2,3) to (1,3,2) one could swap the second pair so Sa = 1, but one could then swap the first pair twice and then Sa = 3. This number (-1)S is of course ±1 and we shall call it the parity of the permutation a, Parity(a) ≡ (-1)S . (B.1.3) Fact: Parity(Ca) = Parity(CAz0) = (-1)S+S = Parity(C-1a) (B.1.4) Proof: Doing permutation CA involves first doing A with its Sa swaps, and then doing C with its Sc swaps, for a total of Sa+Sc swaps. The inverse permutation C-1 obviously involves the same number of swaps as the permutation C. Moving now toward the definition of the determinant of a matrix M, define (Π means "product") Π(a,b; M) ≡ Mab ≡ MabMab ...... Mab = product of n factors . (B.1.5) The notation Π(a,b; M) is easier to deal with than Mab, but we shall use both these notations. Suppose c = Cz0 is some arbitrary permutation of z0. Then: Fact: Π(Ca,Cb; M) = Π(a,b; M) (B.1.6) Proof: Applying the same permutation to both the ak and bk indices of MabMab ...... Mab merely reorders the terms in the product but the product stays the same. Fact: Π(a,b; MT) = Π(b,a; M) (B.1.7) Proof: Π(a,b; MT) = MTabMTab .... MTab = MbaMba .... Mba = Π(b,a; M) . We shall now define the determinant of an n x n matrix M in the following admittedly obscure manner (later we will show that it reduces to more familiar forms), det(M) ≡ Σa Σb (-1)S+S Π(a,b; M) = Σa,b (-1)S+S Mab = Σa Σb (-1)S+S MabMab ...... Mab . (B.1.8) Here Σa means the sum over all permutations a of z0. In (B.1.8) the columns and rows of M are on a completely equal footing. Notice that a and b are in effect dummy summation indices. If we do a↔b, the expression for det(M) is unchanged. Thus one can rewrite (B.1.8) as det(M) = Σb Σa (-1)S+S Π(b,a; M) . (B.1.9) We are now ready for our first determinant theorem: Theorem 1: det(MT) = det(M). Switching rows with columns does not change a determinant. (B.1.10) Proof: det(MT) = Σb Σa (-1)S+S Π(b,a; MT) // (B.1.9) applied to MT = Σb Σa (-1)S+S Π(a,b; M) // (B.1.7) applied to MT = det(M) // (B.1.8) B.2 The permutation group, the permutation tensor ε, and expansions for det(M) Definition. A set of elements {gi} form a group G if : gigj is also in the group (closure) gi-1 exists for each gi in G, where gi-1 is also in G (inverse) (gigj)gk = gi(gjgk) (associative) (B.2.1) Comment: It is implicit in the above definition that the group has some "operation" which gives meaning to gigj, which we shall just think of as "multiplication". When group elements are represented by matrices, that operation is multiplication of those matrices, and that will apply to the permutation group below. Fact: giG = G (the rearrangement theorem) (B.2.2) This says that multiplication of all the elements of a group by an element gi in the group creates a reordering of the group elements. Here G is the set of group elements. Proof. Consider giG = gi [g1, g2....gn] = [gig1, gig2....gign] = set of n elements. Unless two elements are the same, this must exhaust the entire group. How do we know that gig1 and gig2 might not be the same? Apply gi-1 from the left and that would say g1 = g2 which is not the case. Fact: Σg f(g) = Σg f(g1g) if Σg runs over the entire group G (B.2.3) Proof: In Σg f(g1g), as g runs over G, the argument g' ≡ g1g runs over G by the above rearrangement theorem. Thus, the sum Σg f(g1g) is just a reordering of the terms in the sum Σg f(g). It is easy to show that the set of permutations of z0 forms a group and therefore the above facts can be used. For example, the product of two permutations is a permutation, and every permutation clearly has an inverse, and (AB)C = A(BC) for any matrices. Fact (B.2.3) can be written ΣB f(Bzo) = ΣB f(CBz0) // b = Bz0 Here the sum ΣB is over all permutation matrices which correspond to permutations b of z0, and C is some arbitrary permutation matrix associated with some permutation c of z0. An equivalent way of stating the above involves a direct summation Σb over all permutation vectors b of z0 , Σb f(b) = Σb f(Cb) . // permutation sum rearrangement theorem (B.2.4) Throwing in the arbitrary permutation C merely causes a reordering of the sum. This is an extremely powerful and useful fact. Consider then : det(M) ≡ Σa [Σb (-1)S+S Π(a,b; M) ] // (B.1.8) = Σa [Σb (-1)S+S Π(A-1a,A-1b; M)] // (B.1.6) with C = A-1; A-1a = A-1Az0 = z0 = Σa [ Σb Parity(A-1b) Π(z0,A-1b; M) ] // (B.1.4) with C = A-1 and a = b = Σa [ Σb Parity(b) Π(z0,b; M) ] // (B.2.4) with C = A-1 (key step) = Σb Parity(b) Π(z0,b; M) // n! identical terms in Σa = Σa Parity(a) Π(z0,a; M) // rename dummy sum variable = Σa Parity(a) M1aM2a ...... Mna . // (B.1.5) (B.2.5) Since det(MT) = det(M) from Theorem 1 (B.1.10), we can also write this result as det(M) = Σa Parity(a) Ma1Ma2 ...... Man (B.2.6) Proof: det(M) = det(MT) = Σa Parity(a) MT1aMT2a ...... MTna = Σa Parity(a) Ma1Ma2 ...... Man Here then are the last two results: ( recall that Sa is the number of pairwise swaps to get from z0 to a ) det(M) ≡ Σa Parity(a) M1aM2a ...... Mna Parity(a) = (-1)S (B.2.7a) det(M) ≡ Σa Parity(a) Ma1Ma2 ...... Man Parity(a) = (-1)S . (B.2.7b) Definition: The permutation tensor εijk.. (n subscripts) : ε12...n = 1 for any index swap, ε changes sign: ε..i..j.. = - ε..j..i.. therefore, if any two indices are the same, ε = 0 ε..i..i.. = - ε..i..i.. = 0 (B.2.8) Fact: If a is a permutation of z0, then εaa... a = Parity(a) = (-1)S ≡ εa . (B.2.9) Proof: Since indices ai represent a permutation of z0 , it takes Sa swaps to get from εaa... a to ε123...n by the definition of ε. In dense notation, one could say εa = (-1)S εz . Using (B.2.9) in (B.2.7) then gives these two classic det(M) expressions : Theorem 2: det(M) can be represented in these two ways: det(M) = Σaa... a εaa... a M1aM2a ...... Mna (B.2.10a) det(M) = Σaa... a εaa... a Ma1Ma2 ...... Man . (B.2.10b) In dense notation, one could write the above as (a is now a vector), det(M) = Σaεa Mza (B.2.10a)' det(M) = ΣaεaMaz . (B.2.10b)' In most applications, the following simpler notation suffices: det(M) = Σabc..q εabc..q M1aM2bM3c....Mnq (B.2.11a) det(M) = Σabc..q εabc..q Ma1Mb2Mc3....Mqn . (B.2.11b) Theorem 3: For a square matrix M, adding a multiple of one row to another does not change det(M). The same is true for adding a multiple of one column to another. (B.2.12) Proof: (rows) Suppose we replace r3 → r3 + α r2. This says M3i → M3i + α M2i . Eq (B.2.11a) says : det(M') = Σabc..q εabc..q M1aM2b(M3c + α M2c) ....Mnq = det(M) + α Σabc..q εabc..q M1aM2bM2c ....Mnq . Since M1aM2bM2c ....Mnq is symmetric under b↔c while εabc..q is anti-symmetric, the extra α term vanishes. That is to say, sum = Σbc AbcSbc = Σcb AcbScb = Σcb (-Abc)(Sbc) = - sum = 0. Using (B.2.11b) this argument shows that c3 → c3 + α c2 similarly does not alter det(M). Theorem 4: Swapping two rows (columns) of square matrix M causes det(M) → - det(M). (B.2.13) Proof: Let's swap rows 1 and 3 in M to get M'. Then from (B.2.11a), det(M') = Σabc..q εabc..q M3aM2bM1c....Mnq = Σabc..q [- εcba..q] M3aM2bM1c....Mnq // swap a↔c on ε = - Σabc..q εabc..q M3cM2bM1a....Mnq // dummy rename a↔c = - Σabc..q εabc..q M1aM2bM3c....Mnq // reorder product = - det(M) . Using Theorem 1 that det(A) = det(AT), we then know that det(M'T) = - det(MT), so swapping columns 1 and 3 negates det(M). An obvious corollary follows if the two swapped columns have identical data : Corollary 4: If two rows (columns) of square matrix A are the same, then det(M) = 0. (B.2.14) Here is one more well-known determinant theorem which again demonstrates the power of the rearrangement theorem (B.2.4). Theorem 5: det(AB) = det(A)det(B) (B.2.15) Proof: Since we already use A in a = Az0 , we shall prove det(XY) = det(X)det(Y) to avoid overloading symbols. Note that Parity(a) Parity(b) = (-1)S(-1)S = (-1)S+S = Parity(Ab) from (B.1.4). Then : det(X)det(Y) = [ Σa Parity(a) Π(z0,a; X)] [ Σb Parity(b) Π(z0,b; Y)] // (B.2.5) twice = Σa [ Σb Parity(a) Parity(b) Π(z0,a; X) Π(z0,b; Y) ] = Σa [ Σb Parity(Ab) Π(z0,a; X) Π(Az0,Ab; Y) ] // (B.1.6) with C = A = Σa [ Σb Parity(Ab) Π(z0,a; X) Π(a,Ab; Y) ] // Az0 = a = Σa [ Σb Parity(b) Π(z0,a; X) Π(a,b; Y) ] // (B.2.4) with C = A = Σb Parity(b) Σa Π(z0,a; X) Π(a,b; Y) // move Σa = Σb Parity(b) Σa Π(z0,b; XY) // see below = det(XY) . // (B.2.5) The idea here is to get a in the right place on both Π's so that Σa Π(z0,a; X) Π(a,b; Y) = Σ aa... a X1aX2a ...... Xna YabYab ...... Yab = Σ aa... a (X1aYab)(X2aYab).....(XnaYab) = (XY)1b (XY)2b ... (XY)nb = Π(z0,b; XY) which in dense notation one would write as Σa XzaYab = (XY)zb . Corollary 5: det(ABC) = det(A)det(B)det(C) and so on. (B.2.16) Proof: det(ABC) = det(A[BC]) = det(A)det(BC) = det(A)det(B)det(C) . It is not hard to slightly generalize the results of Theorem 2 to obtain : Theorem 2A: If b is a permutation of z0, then det(M) can be represented in these two ways: det(M) = εbb... b Σaa... a εaa... a MbaMba ...... Mba (B.2.17a) det(M) = εbb... b Σaa... a εaa... a MabMab ...... Mab . (B.2.17b) In dense notation, one could write the above as (a and b are now vectors), det(M) = εbΣaεaMba (B.2.18a) det(M) = εbΣaεaMab . (B.2.18b) Since εb2 = 1, one can move εb to the left side of these equations, so in the dense notation, εbdet(M) = ΣaεaMba (B.2.19a) εbdet(M) = ΣaεaMab . (B.2.19b) Equations (B.2.19) are valid even if b is not a permutation of z0. In this case one has 0 = 0. For the special case b = z0 Theorem 2A reduces to Theorem 2 since εz = ε12...n = 1. Proof of Theorem 2A: We shall prove (B.2.17a) and then (B.2.17b) follows by taking M→MT and using det(MT) = det(M). det(M) = Σa Parity(a) Π(z0,a; M) // line 6 of (B.2.5) = Σa Parity(a) Π(Bz0,Ba; M) // (B.1.6) with C = B = Σa Parity(Ba) Parity(b) Π(Bz0,Ba; M) // Parity(Ba) Parity(b) = Parity(a) = Σa Parity(a) Parity(b) Π(Bz0,a; M) // (B.2.4) that Σa f(Ba) = Σa f(a) = Parity(b) Σa Parity(a) Π(b,a; M) // b = Bz0 = εbb... b Σaa... a εaa... a MbaMba .... Mba // (B.1.5) and (B.2.9) = εb Σa εa Mba . // previous line in dense notation (B.2.20) Here it is assumed that B and hence b is associated with a permutation of z0, In this case, (εb)2 = 1 so one can move εb to the left side to get εbdet(M) = Σa εa Mba . (B.2.21) In this form, the equation is valid whether or not b is a permutation of z0 . If in b = (b1,b2...bn) the bi are arbitrary elements of the set {1,2,...n} where two or more bi are the same (that is, b is not a permutation of z0), then Σa εaa... a MbaMba .... Mba = 0 by symmetry. For example, if b1 = b2 then S ≡ Σa εaa... a MbaMba .... Mba = Σa εaa... a MbaMba .... Mba // rename dummy indices a1 ↔ a2 = Σa [-εaa... a ] MbaMba .... Mba // (B.2.8) for ε and slide Mba to the right = - Σaεaa... aMbaMba .... Mba = - S S = 0 . (B.2.22) So in this case the right side of (B.2.21) is zero. But from (B.2.8) εb = 0 so the left side is also zero. Reader Exercise: Prove that εi... idet(M) = ΣA parity(A) M1,A(i)M2,A(i)......Mn,A(i) . (B.2.23) Proof in dense notation: εidet(M) = εiΣa εa Mz,a // this is εi times usual expansion (B.2.7a) = parity(I) ΣA parity(A) Mz,Az // a = Az0 = parity(I) ΣA parity(AI) Mz,AIz // rearrangement theorem ΣAf(A) = ΣA f(AI) = ΣA parity(A) Mz,Ai // i = Iz0 QED B.3 Minors, Cofactors, and the Cofactor Expansions of det(M) Definition: The minor of a matrix element Mrs is the determinant of the submatrix of M obtained by crossing out the rth row and the sth column. It is a challenge, however, to write this out in symbols. Let's start with a more detailed version of (B.2.11a), where the ai are column summation indices, det(M) = Σaaaaa...a εaaaaa...a M1aM2aM3aM4aM5a ...... Mna . (B.3.1) We shall make the following conjecture for the form of minor(M23) minor(M32) = (-1)3-2 Σaaaa...a εaa2aa...a M1aM2aM4aM5a ...... Mna (B.3.2) Compared to det(M) shown in (B.3.1), we have made these changes : Removed the factor M3a ( since row-3 matrix elements cannot appear in minor(M32) ) Removed the sum over a3. Replaced a3 by the number 2 on the ε tensor. added a sign factor (-1)3-2. Notice that since there is a 2 on the ε tensor, any time a summation index ai = 2 there is no contribution since then the ε tensor has two indices the same, so in effect the value 2 has been removed from all the residual summations ai. That is good, since column 2 is supposedly "crossed out" in minor(M32). The factor (-1)3-2 is added so that the "diagonal term" in the minor will be positive. This sign (-1)3-2 gets used up if we slide the "2" to its natural position (position 2) on the ε tensor, minor(M32) = Σaaaa...a εa2aaa...a M1aM2aM4aM5a ...... Mna . (B.3.3) The diagonal term in minor(M32) is then positive, matching the mechanical crossing-out method, ε12345..n M11M22M44M55 ...... Mnn = + M11M22M44M55 ...... Mnn . (B.3.4) A more compact notation for (B.3.2) would be minor(M32) = (-1)3-2 Σa,i≠3 εa,a=2 Πi≠3 (Mi,a) . (B.3.5) Starting over, we could show similarly that minor(M42) = (-1)4-2 Σa,i≠4 εa,a=2 Πi≠4 (Mi,a) . (B.3.6) We now have a sign factor (-1)4-2 because the "2" on ε has to be slid 2 positions to get to its natural location (position 4 on ε). More generally we can say minor(Ms2) = (-1)s-2 Σa,i≠s εa,a=2 Πi≠s (Mi,a) (B.3.7) where the slide is now s-2 places. Still more generally we find, replacing 2 by r, minor(Msr) = (-1)s±r Σa,i≠s εa,a=r Πi≠s (Mi,a) . (B.3.8) This then is our "best form" expression for a minor of matrix element Msr. Either sign will do, since (-1)s-r = (-1)s-r(-1)2r = (-1)s+r since (-1)2r = 1 . Choosing the + sign in (B.3.8) and putting (-1)s+r on the left side we get, (-1)s+r minor(Msr) = Σa,i≠s εa,a=r Πi≠s (Mi,a) . (B.3.9) The left side here is called the cofactor of Msr, written cof(Msr). Thus we have shown that cof(Msr) ≡ (-1)s+r minor(Msr) = Σa,i≠s εa,a=r Πi≠s (Mi,a) . (B.3.10) Fact: Neither minor(Msr) nor cof(Msr) are functions of the Msr matrix element of M! In (B.3.10) this is so because: (1) row s is excluded in Πi≠s(Mi,a); (2) ai = r is excluded by the factor εa,a=r . More intuitively, this is so because to get minor(Msr) we "cross out" row s and column r. Thus, = = 0 for any pair r,s in 1,2...n . (B.3.11) Finally, suppose we replace r with an integer which we call as. Doing this causes εa,a=r → εa,a=a = εa = εaaaa...a = the normal ε tensor form . (B.3.12) Then (B.3.10) becomes the following, cof(Msa) ≡ (-1)s+a minor(Msa) = Σa,i≠s εa Πi≠s (Mi,a) . (B.3.13) Now start again with det(M) of (B.3.1) and rewrite it in our compact notation: det(M) = Σaaaaa...a εaaaaa...a M1aM2aM3aM4aM5a ...... Mna = Σa εa Πi(Mi,a) // next, extract the as sum and its Msa factor : = Σa Msa [Σa,i≠s εa Πi≠s (Mi,a)] // next use (B.3.12) to get = Σa Msa [cof(Msa)] // next, change summation index from as to n = Σn Msn cof(Msn) . // valid for any s in 1,2...n (B.3.14) This is the cofactor expansion of det(M) where one "works across row s" and n is a column index. Another form of this expansion is det(M) = det(MT) = Σn MTsn cof(MTsn) = Σn Mns cof(Mns) (B.3.15) and in this form one "works down column s" where n is now a row index. We have just proven : Theorem 6 (Cofactor Expansions): det(M) = Σn Msn cof(Msn) = Σn (rs)n cof(Msn) work across row s s = 1,2....n (B.3.16a) det(M) = Σn Mns cof(Mns) = Σn (cs)n cof(Mns) work down column s s = 1,2....n (B.3.16b) The term "cofactor" presumably arose because each sum term consists of a "factor" Mij and a "co-factor" which cooperates with the factor to make the sum, as coauthors cooperate to write a book. A common notation is to define Mij ≡ cof(Mij) and then the cofactor expansions become det(M) = Σn MsnMsn det(M) = Σn MnsMns . Although compact, this is not so handy for tensor notation with its up and down indices (see Lucht Ref). B.4 Expressions for the inverse matrix M-1 and Cramer's Rule Consider the expansion (B.3.16a) working across row s, det(M) = Σn Msn cof(Msn) . (B.4.1) Suppose we replace the elements of row s (Msn) with the elements of some other row r in M (Mrn). In doing so we have created a new matrix, call it M'. Notice that cof(M'sn) = cof(Msn) because, although going from M to M' we have altered row s, we have not altered cof(Msn) since this depends only on the entries in all the rows other than row s (think "crossing out" row s for minor(Msn) ). See Fact (B.3.11) and text above. Therefore we find, det(M') = Σn M'sn cof(M'sn) = Σn Mrn cof(Msn) . (B.4.2) But by Corollary 4 (B.2.14) det(M') = 0 since two rows of M' are the same. Thus 0 = Σn Mrn cof(Msn) . r ≠ s (B.4.3) Combining this with (B.4.1) gives, Σn Mrn cof(Msn) = det(M)δr,s . (B.4.4) Now for clarity define a cofactor matrix C in this manner Csn ≡ cof(Msn) . (B.4.5) In matrix notation, we could define the matrix cof(M) to be matrix C and then C = cof(M) Csn = [cof(M)]sn = cof(Msn) . (B.4.6) Now (B.4.4) says ΣnMrn Csn = det(M)δr,s or ΣnMrn CTns = det(M)δr,s and finally in matrix notation, MCT = det(M) 1 or M [ ] = 1 . (B.4.7) Therefore we find this classic square matrix inversion formula, Theorem 7: M-1 = = = . (B.4.8) To verify the last equality in (B.4.8) consider : = [cof(M)]Tns = [cof(M)]sn // meaning of transpose = cof(Msn) // (B.4.6) = cof(MTns) = [cof(MT)]ns // (B.4.6) applied to MT and therefore we have this matrix identity, [cof(M)]T = [cof(MT)] . (B.4.9) Using (B.4.8) it is trivial to solve a non-singular (detM≠0) system of linear equations y = Mx : y = Mx x = M-1 y = cof(MT) y = [cof(M)]T y . (B.4.10) In components, xs = Σn ([cof(M)]T)sn yn = Σn yn [cof(M)]ns = Σn yn cof(Mns) . (B.4.11) Now recall the cofactor expansion (B.3.16b), det(M) = Σn Mns cof(Mns) = Σn (cs)n cof(Mns) . (B.4.12) If one replaces column cs in M by y one gets det(M[cs→ y]) = Σn yn cof(Mns) // (B.4.12) = det(M) xs . // (B.4.11) (B.4.13) Solving this for xs we obtain another classic result, Theorem 8 (Cramer's Rule, 1750): // Gabriel Cramer (1704-1752), Swiss, made it to age 47 y = Mx xs = . (B.4.14) Comment: Names of Matrices Historically the matrix CT = [cof(M)]T = [cof(MT)] was called the adjoint of matrix M and was often denoted by and then (B.4.8) reads M-1 = /|M|. But this then conflicted with another usage, namely the matrix M† ≡ (MT)* = (M*)T being the adjoint of M (the conjugate transpose of M). When M† = M, M is said to be self-adjoint and has significance in quantum (matrix) mechanics: the quantum operators of all physical observables are self-adjoint and thus have real eigenvalues. Nowadays the matrix [cof(M)]T is called the classical adjoint of M, while M† is then the Hermitian adjoint (or Hermitian conjugate) of M and is sometimes written MH. If M† = M then M is said to be Hermitian. Historically when [cof(M)]T was the adjoint of M, M† was called the associate of M. The classical adjoint sometimes appears as the adjugate or adj(M). Transpose matrices MT are often denoted . On a related matter, some older authors used the word minor to refer to what we now call a cofactor. See for example Morse & Feshbach page 509. These authors never use the word cofactor. Appendix C: The Intersection of Constraint Surfaces In the proof of Theorem 3 in Section 6.2 (a) we rely on our ability, at least in theory, to eliminate x1, x2, x3 from the three constraint equations shown in (6.2.3). a(x1, x2, x3, x4, x5, x6) = 0 x1 = X1(x4, x5, x6) b(x1, x2, x3, x4, x5, x6) = 0 x2 = X2(x4, x5, x6) c(x1, x2, x3, x4, x5, x6) = 0 x3 = X3(x4, x5, x6) . (6.2.3) How does one know this is possible? The equations might be very complicated, some of the coordinates might not appear in some of the equations, and so on. We look first at three examples in E3 then in Example 4 we return to our question. Example 1: Consider these two constraint equations in E3 : a(x1,x2,x3) = x12 + x22 + x32 - 22 = 0 b(x1,x2,x3) = (x1-2)2 + x22 + x32 - 22 = 0 . (C.1) Each constraint represents a sphere of radius 2 and so is a 2-dimensional smooth surface in E3. The first sphere is centered at the origin, while the second has its center at (2,0,0). Subtracting the first equation from the second gives -4x1+4 = 0 so x1= 1, then inserting this into both equations gives 1 + x22 + x32 - 22 = 0 1 + x22 + x32 - 22 = 0 x22 + x32 = 3 . (C.2) The intersection of the two constraint surfaces is a circle, a 1-dimensional smooth surface in E3. Given a(x1,x2,x3) = 0 and b(x1,x2,x3) = 0, is it possible to eliminate x1 and x2 from the two constraint equations? The answer is yes, as follows, x1 = X1(x3) = 1 x2 = X2(x3) = ± . (C.3) The following schematic drawing shows the intersection surface in red, (C.4) Notice in this example that there are two possible solutions for x2. One can regard the intersection surface (the circle x22 + x32 = 3) as having two pieces (front half, back half) and the ± sign selects one of these pieces. It is easy to imagine more complicated examples for two constraints in E3 where perhaps one or both of the constraint surfaces contain disjoint pieces (e.g. a hyperboloid), and where the intersection surface also has several disjoint pieces. Example 2: Consider these two constraint equations in E3 : a(x1,x2,x3) = x12 + x22 + x32 - 22 = 0 b(x1,x2,x3) = x22 + x32 - 12 = 0 . (C.5) The second equation is missing the x1 coordinate. The first equation describes the same origin-centered radius 2 sphere of the previous example. The second equation is that of a circle of radius 1, but as a function of three variables it is in fact a cylinder whose axis is the x1 axis. If x2 and x3 lie on the circle, any value of x1 satisfies the second equation. Missing coordinates result in surfaces which are "extruded" in the dimensions of the missing coordinates. Subtracting the second equation from the first gives x12 - 3 = 0 so x1 = ±, then inserting this into both equations gives 3 + x22 + x32 - 22 = 0 x22 + x32 - 12 = 0 x22 + x32 = 1 . (C.6) Given a(x1,x2,x3) = 0 and b(x1,x2,x3) = 0, is it possible to eliminate x1 and x2 from the two constraint equations? The answer is yes, as follows, x1 = X1(x3) = ± x2 = X2(x3) = ± . (C.7) The following schematic drawing shows the intersection surface in red, (C.8) In this example there are two possible solutions for x1 and two for x2 so overall there are four solutions. The intersection surface consists of the two circles each having a front half and a back half. Example 3: Consider these two constraint equations in E3 : a(x1,x2,x3) = x12 + x22 + x32 - 22 = 0 b(x1,x2,x3) = x12 + x22 - 12 = 0 . (C.9) Now x3 is missing from the second equation instead of x1. Subtracting the second equation from the first gives x32 - 3 = 0 so x3 = ±, then inserting this into both equations gives x12 + x22 + 3 - 22 = 0 x12 + x22 - 12 = 0 x12 + x22 = 1 . (C.10) Given a(x1,x2,x3) = 0 and b(x1,x2,x3) = 0, is it possible to eliminate x1 and x2 from the two constraint equations? The answer is yes, as follows, where α is an arbitrary real parameter in [-1,1], x1 = X1(x3 = ±) = α -1 ≤ α ≤ 1 x2 = X2(x3 = ±) = ± . // the two ± signs are independent (C.11) In the previous two examples the argument x3 was a free parameter whose variation mapped out the smooth constraint intersection surface piece(s). In Example 3 x3 is fixed at ± and a new parameter α must be introduced to map out the intersection surfaces. There are again four solution half-circles. The following schematic drawing shows the intersection surface in red, (C.12) Example 4: Now consider these three constraint equations in E6 : a(x1, x2, x3, x4, x5, x6) = 0 b(x1, x2, x3, x4, x5, x6) = 0 c(x1, x2, x3, x4, x5, x6) = 0 . (6.2.2) (C.13) We assume that each equation describes a smooth surface of dimension 5 in E6 (each is a hypersurface) and each surface may have several pieces. In a problem with constraints, one assumes that there is some non-null surface of intersection of all the constraint surfaces. This intersection surface may have multiple pieces and each piece (barring pathological cases) is itself a smooth surface of dimension 3 in E6. A candidate solution point r must lie on one of these intersection surface pieces. In general, each constraint knocks down the dimension of the intersection surface by one degree of freedom. So let us assume that x4, x5, x6 are the last three components of some point(s) on the overall intersection surface. Each such point of course has some x1, x2, x3 components. If x4, x5, x6 are varied slightly, x1, x2, x3 will also vary slightly, and this is what we mean by writing the functions x1 = X1(x4, x5, x6) x2 = X2(x4, x5, x6) x3 = X3(x4, x5, x6) . (6.2.3) (C.14) If the intersection surface has multiple pieces which have the same x4, x5, x6 value, then any of the three functions Xn above may be multi-valued, as occurred in our earlier examples. Conversely, if we select a triplet x4, x5, x6 for which there are no points on the intersection surface, the solution x1, x2, x3 does not exist. We don't care about such points in E6. So this then is what we mean in Section 6.2 (a) when we say above (6.2.3) that one can use the three constraint equations to eliminate the three variables x1, x2, x3. In the concluding equations (6.2.11) one can regard the functions X1, X2, X3 as being any of the multi-valued functions just discussed if the intersection surface has multiple pieces. For example, we had in (6.2.11) that = part of (6.2.11) (C.15) and here X14 refers to (∂/∂x4)X1(x4, x5, x6) evaluated at coordinates x4, x5, x6 for some candidate solution point r on the intersection surface. The fact that the matrix in (C.15) must have zero determinant is not affected by the possible existence of multiple solution X1, X2, X3 functions. Notice that the derivatives appearing in the matrix are unaffected by the possibility of multi-valued functions for X1, X2, X3. In the same manner as above, we can "eliminate" any triplet of variables xi, xj, xk ( i ≠ j ≠ k) from the three constraint equations (C.13). In the general case where there are S-1 constraints and r has N components x1,x2...xN with N > S, the dimensionality of the intersection surface is N - (S-1) in EN. The conclusions of the Theorem 3 proof outlined in Section 6.2 (b) are similarly not affected by the possible existence of an intersection surface having multiple pieces and the functions Xn possibly having multiple values. Footnote: Why is a(x1, x2, x3, ...xN) = 0 an (N-1)-dimensional surface in EN ? (C.16) This fact probably seems obvious to the reader and is certainly obvious in the first three examples above. Here we attempt a simple "engineering" explanation. We assume that a constraint equation is a "smooth" equation, meaning it is continuous and differentiable in all its arguments except perhaps at certain isolated locations which can be handled on an ad hoc basis either by fiat or by taking limits. Consider that, since a(r) = constant (namely 0), da = 0 so 0 = da = Σi=1N aidxi , // ai = ai(r) = ai(x1, x2, x3, ...xN) (C.17) where ai is usual means ∂a/∂xi. Let r be some point for which a(r) = 0. At this point we shall assume that there exists at least one as(r) which is non-zero. If all ai(r)= 0, we show below that r is a point on a surface which has a null normal vector, which is not possible, so such an r value could not lie on a(r) = 0. Then (C.17) may be written dxs = (1/as) Σi≠s aidxi . (C.18) Create a coordinate system whose origin is located at this point r for which a(r) = 0, and whose axes are aligned with those of EN. Now imagine an arbitrary tiny displacement of all the dxi other than dxs. For this set of N-1 dxi there is only one possible dxs which causes the point r + dr to satisfy the equation a(r+dr) = a(r) + da = 0 + 0 = 0, where dr ≡ (dx1, dx2.....dxN). That one possible dxs value is that given by (C.18). Imagine repeating this process for a continuum of values for the dxi other than dxs, and in each case we obtain the unique solution dxs from (C.18). We can think of the xs axis as being "vertical" and all the other axes being "horizontal" inasmuch as they are all perpendicular to the xs axis. The vectors dr generated in this manner comprise a tiny patch of a "plane" of dimension N-1 (a hyperplane) in EN which contains the point r. This is so because Σi=1N aidxi = a dr = 0 is the equation of a plane in EN passing through our origin at r with a being that plane's normal vector. More generally, r = d is the equation of a hyperplane in En having a unit normal and whose closest approach to the origin is distance d. In Thus at a point r satisfying a(r) = 0 we have constructed a tiny neighborhood of nearby points r which also satisfy a(r) = 0. This neighborhood is a patch of a hyperplane in EN which is certainly a piece of "surface" in EN having dimension N-1. By repeating this process using a mesh of points ri satisfying a(ri) = 0, we then map out a triangulated surface of dimension N-1 in EN. In the limit the mesh size goes to 0, we arrive at a smooth surface of dimension N-1 in EN and that surface is a(r) = 0. The tiny planar patch at any point on the surface is part of the "tangent plane" to the surface at that point. The constraint a(r) = 0 is assumed to be "locally smooth" in the immediate region of any point r on the operational constraint surface, so a small local planar region (an open set) can be constructed around any such point. This is the basic idea of a surface being a manifold M, and the set of vectors in the tangent plane of a point r on M comprise the "tangent space" of M at r, usually denoted by TrM. Appendix D: Lagrange Multipliers in Classical Mechanics Outline of this Appendix showing the nearest equation number: Constraints and virtual displacements (D.1) Assumption that constraints do no virtual work (D.11) Comment on internal forces (D.13) An Application of Lagrange Multipliers (D.13) Summary of the above Lagrange Multiplier Application (D.25) Generalized coordinates, generalized forces, and Lagrange's Equations (D.27) Footnote: Derivation of the result (D.31) used above (D.40) Lagrange Equations with non-holonomic constraints (D.49) The Case of C holonomic constraints treated as if they were non-holonomic (D.55) Footnote: Carry out the δS = 0 functional variation of the action (D.56) Our main focus in this document is the use of Lagrange Multipliers to find candidates r for which a scalar function f(r) is stationary, df = 0, subject to constraints. In this Appendix, however, we consider applications of Lagrange Multipliers which relate to classical mechanics for a system of N particles in the presence of constraints. In this case the function f(r) with r lying in EN is replaced by a functional f(φ) where φ lies in a space of functions. The goal is to find a function φ which is a stationary point of the functional, and one writes this as δf(φ) = 0. A certain amount of background information is required to put this application into context, which we now present. Constraints and virtual displacements Let rk(t) be the position of the kth particle of an N particle system in 3 dimensional space. At any fixed time t, if the particles are unencumbered with constraints, the variables (r1, r2..... rN) can be regarded as a single point in the space E3N and the "configuration space" of the system indicated by (r1, r2..... rN) is all of this space E3N. All points in E3N are "legal" points for the system. A holonomic constraint has the form a(r1, r2..... rN, t) = 0. (D.1) At a given time t, this equation represents a surface of dimension 3N-1 in EN. If there are s such holonomic constraints ai(r1, r2..... rN, t) = 0 i = 1,2...s (D.2) then at time t the system vector (r1, r2..... rN) is constrained to lie on a surface of dimension 3N-s in EN, as discussed in Appendix C, so each extra constraint lowers the operating surface dimension by 1. One can think of the term holonomic as meaning that the constraints are functions of "whole" coordinates like r2 as opposed to differential coordinates like dr2. The constraints which appear in (1.7) are holonomic constraints of the form a(x1, x2, ... xn) = 0. A non-holonomic constraint is one that is not holonomic. An inequality constraint like a(r1, r2..... rN, t) < 0 which might bound the particles to one side of a surface is an example of a nonholonomic constraint, but our interest lies more in nonholonomic constraints of the form Σk=1N bk drk + btdt = 0 (D.3) in which differentials rather than "whole" coordinates appear. The "coefficients" bk and bt can in general be functions of (r1, r2..... rk, t). If it happens that such a constraint can be integrated by some hook or crook to give an equation of the form a(r1, r2..... rk, t) = 0, then it is regarded as being a holonomic constraint, so a non-holonomic constraint must be non-integrable. If there are m such non-holonomic constraints imposed on a system, one could write them as Σk=1N Aik drk + Aitdt = 0 i = 1,2...m or (D.4) Σk=1N Aik k + Ait = 0 i = 1,2...m . Here bolded Aik represents 3 matrices each of which is m x N (rows x columns). Since these constraints are non-integrable, one cannot really described them in terms of surfaces in E3N. If one differentiates the holonomic constraints (D.2) one gets Σk=1N [(k)ai] drk + [∂tai] dt = 0 i = 1,2...s or (D.5) Σk=1N [(k)ai] k + [∂tai] = 0 i = 1,2...s where (k)≡ (∂/∂(rk)1, ∂/∂(rk)2, ∂/∂(rk)3) is a gradient with respect to the components of rk. These equations have the same form as those in (D.4), but these are holonomic because they are integrable to give ai(r1, r2..... rN, t) = 0. For convenience below, we shall now combine (D.4) and (D.5) into a single set of equations, Σk=1N Bik drk + Bitdt = 0 i = 1,2....C // C = s+m or (D.6) Σk=1N Bik k + Bit = 0 i = 1,2....C , where Bik = Aik Bit = Ait for i = 1,2...s non-holonomic Bik = (k)ai Bit = ∂tai for i = s+1, s+2....C holonomic . (D.7) In general, the functions ai, Bik and Bit are all functions of (r1, r2..... rk, t). In the above we have followed the notation of Ray and Shamanna (2006) and we continue our path in the manner they propose. Suppose the set of N differentials {drk} satisfies all of equations (D.6) for some time interval dt. Such a set {drk} is then called a set of allowed displacements. Imagine that {dr'k} is some other distinct allowed displacement set. Elements of the difference set defined as {δrk} = {drk} - {dr'k} k = 1,2...N (D.8) are then called virtual displacements. This is how Ray and Shamanna define the vague term virtual displacement which appears in many texts (about which they have much to say in their Section I A). It follows then that for any virtual displacement set {δrk} equations (D.6) take this form Σk=1N Bik δrk = 0 i = 1,2....C (D.9) simply because the last term in (D.6) cancels out when one writes δrk = drk - dr'k. Notice that the δrk are not arbitrary differential displacements. They are differences of allowed displacements. Meanwhile, Newton's Law for a system of N particles subject to constraints has the form mk = Fk + Rk k = 1,2...N (D.10) where Rk is the sum of all constraint forces applied to particle k and Fk is the sum of all other forces applied to particle k. Assumption that constraints do no virtual work Next, assume that ( R&S refer to this as being an "ideal constraint" situation), 0 = Σk=1N Rk δrk . (D.11) In Goldstein this equation corresponds to setting the second term in (1-40) to zero. The main justification for making this assumption is that it is valid for particles which form a rigid body and for many other constraint situations. There seems to be no general justification of this equation for all constraint situations, and it is certainly not valid if friction is present. The equation can be interpreted as saying that the total virtual work done by all the constraint forces on all the particles of the system vanishes. As a first simple example, consider two particles 1 and 2 independently sliding down a frictionless static ramp. In this case for particle 1 the constraint force R1 is the normal force of the ramp acting on the particle, and this is clearly perpendicular to dr1 for any allowed displacement set {drk}, and thus R1 is also perpendicular to any δr1 which is part of any virtual displacement set {δrk}. In this case we have R1 δr1 = 0 and R2 δr2 = 0 so the terms in (D.11) are separately zero. A second example is more interesting. Particles 1 and 2 of arbitrary masses slide down a one dimensional ramp but are glued to the two ends of a massless stick so they comprise a rigid body. The upper mass is attached to a string which provides a constant tension T. Here is a picture, (D.12) Particle 2 exerts a constraint force T12 on particle 1 through the stick, and Particle 1 exerts a constraint force T21 on particle 2. Since there is only one force of tension (or compression) in such a stick, T12 = T21. All allowed displacement sets {dr1, dr2} are of the form {dr, dr}, and therefore all virtual displacement sets are of the form {δr1, δr2} = {δr, δr}. In (D.12) we have drawn the two normal forces Ni and the two total constraint forces Ri. It is no longer true that R1 δr1 = 0 and R2 δr2 = 0 as the picture shows. However, it is true that Σk=12 Rk δrk = [ Σk=12 Rk ] δr = 0 because R1 and R2 have equal and opposite components along the ramp, so equation (D.11) is valid for this example. Now suppose the ramp is translating in some direction not along the ramp. The displacements dr1= dr2 are then no longer along the ramp so that Σk=12 Rk drk ≠ 0. However, any virtual displacement set would still have δr1 = δr2 along the ramp and then again Σk=12 Rk δrk = 0. Remember that a virtual displacement set is the difference of any two allowed displacement sets, and in this subtraction the effect of the ramp translation cancels out. So equation (D.11) is still valid. Other examples are given in Ray and Shamanna Section III. Reader Exercise: show for (D.12) that T12 = T m2 / (m1+m2) and that this makes sense if m1 >> m2 and vice versa. We proceed then assuming our system of interest respects equation (D.11). Comment on internal forces In (D.10) we have classified the forces acting on particle k into two groups and we write Rk + Fk, where Rk are "constraint" forces and the Fk are any "other" forces. Goldstein refers to the Fk forces as "applied" forces in (1-39), while Ray and Shamanna call them "external" in Sec IV. Consider a system consisting of N masses all connected by a network of massless springs. The force acting on particle k due to all the springs to which it is attached would be best classified in the Fk "other" category. Were we to classify this force into the Rk group, we could not use the no-constraint-work property (D.11) since spring forces do work as the springs compress and expand. An example of such a system is the Solar System where the spring forces are represented by gravity. It would seem a misnomer to classify the internal forces in this case as "applied" or "external". However, if all the springs are slowly adjusted until they become infinitely stiff, in that limit it would be better to classify the internal spring forces into the Rk constraint forces group, since these springs do no work and (D.11) applies. The system is then a "rigid body" and in this case, with the internal forces classified as Rk constraint forces, one benefits from the many simplifications which arise for rigid body mechanics. The internal forces in our spring example fall into the category where Fij = - Fji and Fij is in the direction ri- rk . When this is not the case, the rigid body limit does not apply, but the internal forces can still be classified as "constraint" plus "other". An Application of Lagrange Multipliers Recall the C = m+s constraint equations (D.9), Σk=1N Bik δrk = 0 i = 1,2...C . // C ≡ m+s = total number of constraints (D.9) Multiply each of these C equations by a Lagrange Multiplier λi and then add the equations to get 0 = Σi=1C λi [Σk=1N Bik δrk] = 0 = Σk=1N [ Σi=1C λi Bik ] δrk . (D.13) Now subtract equation (D.13) from (D.11) to get 0 = Σk=1N [ Rk - Σi=1C λi Bik] δrk . (D.14) We know that the N vectors in the set {δrk} are not linearly independent because they must satisfy the equations (D.9), so we cannot claim from (D.14) that [ Rk - Σi=1C λi Bik] = 0 for each k. But now write out equation (D.14) in full detail, showing all vector components, 0 = Σk=1N Σj=13 [ (Rk)j - Σi=1C λi (Bik)j] (δrk)j . (D.15) Next, relabel the 3N terms in this sum using a single index n defined in this way (k,j) = (1,1), (1,2), (1,3), (2,1), (2,2), (2,3),..........(N,1), (N,2), (N,3) n = 1, 2, 3, 4, 5, 6 , 3N-2, 3N-1, 3N so (D.16) n = 3(k-1)+j k = 1+ Int[(n-1)/3] j = 1 + Rem[(n-1)/3] . Then define (here the n value is implied by the k,j labels as per above) xn ≡ (rk)j Rn ≡ (Rk)j Bin ≡ (Bik)j . (D.17) Equation (D.15) can then be expressed as 0 = Σn=13N [ Rn - Σi=1C λi Bin] δxn (D.18) and (D.9) becomes 0 = Σk=1N Σj=13 (Bik)j (δrk)j or 0 = Σn=13N Bin δxn i = 1,2...C . (D.19) Once again, we cannot set the square brackets in (D.18) to 0 because the 3N coordinates δxn are not linearly independent due to the C equations (D.19). At any given time t, due to these equations (D.19), there must exist at least one subset of the 3N variables δxn which are independent and the remaining C δxn are dependent. Assume that the dependent variables have indices n = n1, n2,...nC. Consider then the following set of C equations involving the dependent-term square brackets in (D.18), [ Rn - Σi=1C λi Bin] = 0 n = n1, n2,...nC . (D.20) This represents C linear equations in the C parameters λi so, when all is said and done, there must exist a set of values {λi} which make these C equations be true. We assume these values will be λi(t) and this is justified below when the full set of equations is considered. Now the terms in (D.18) indicated by n = n1, n2,...nC all vanish due to (D.20), so (D.18) can be written, 0 = Σn≠n,n,....n [ Rn - Σi=1C λi Bin] δxn . (D.21) But all the δxn appearing in (D.21) are independent, so the square brackets here must also vanish. We then end up with all the square brackets in (D.18) being 0 (assuming the correct solution values of λi), [ Rn - Σi=1C λi Bin] = 0 n = 1,2....3N or Rn = Σi=1C λi Bin n = 1,2....3N or (Rk)j = Σi=1C λi (Bik)j k = 1,2..N j = 1,2,3 or Rk = Σi=1C λi Bik k = 1,2...N . (D.22) Thus the constraint forces are certain linear combinations of the Bik constraint functions shown in (D.7). Meanwhile, Newton's Law (D.10) says that mkak = Fk + Rk where the Fk are the non-constraint forces for the problem. Inserting (D.22) gives mkk = Fk + Σi=1C λiBik . Bik = (D.23) The problem is then summarized in the following set of equations from (D.23) and (D.6), mkk(t) = Fk(r1, r2..... rN, t) + Σi=1C λi(t)Bik(r1, r2..... rN, t) k = 1..N 3N equations k(t) = vk(t) k = 1..N 3N equations Σk=1N Bik(r1, r2..... rN, t) k(t) + Bit(r1, r2..... rN, t) = 0 i = 1,2....C . C equations (D.24) This can be regarded as a set of 6N+C scalar first-order ODEs in the single variable t for the 6N+C unknown scalar functions rk(t), vk(t) and λi(t). Although the first 3N equations happen to be linear in the λi(t), the equations are in general non-linear in the rk(t) so this is then a non-linear system of ODE's. Nevertheless, a well known existence theorem (e.g., Ince Section 3.3) states that, provided the Fk, Bik and Bit functions are continuous and differentiable in all their arguments, a unique solution exists for any set of initial values rk(0), vk(0) and λi(0). As (D.22) shows, the λi(0) are related to the constraint forces at time 0. The uniqueness of the solution agrees with our intuition that a physical system will evolve in a unique manner. Finding an analytic form for the solution might not be possible, but a numeric form can always be obtained. Summary of the above Lagrange Multiplier Application We have used a set of Lagrange Multipliers as an aid to solving a mechanics problem involving N particles with C constraints. As already noted, the above presentation is based on Section IV of Ray and Shamanna. The steps were as follows, considered more generically: 1. One has a sum Σn=1NRnδxn = 0. (D.25) 2. One also has a set of sums Σn=1NBinδxn = 0 for i = 1,2...C with C < N (constraints in the above case). This implies that one can find C of the δxn which are linearly dependent on the other δxn. 3. If λi are C arbitrary factors, certainly Σi=1Cλi(Σn=1NBinδxn) = 0 based on 2. These λi are "Undetermined Lagrange Multipliers". 4. Subtract sum 3 from sum 1 to get the new sum Σn=1N[ Rn - Σi=1Cλi Bin] δxn = 0. 5. The C λi are selected so that [..] = 0 in the C dependent δxn terms of this sum. The remaining sum has only independent δxn terms so [..] = 0 for all those terms as well. Then all [...] = 0 in the sum of step 4. 6. The conclusion is that [ Rn - Σiλi Bin] = 0 for all n, assuming the solution λi values of step 5. In contrast with our main topic of finding stationary points df = 0 of a function f(r), in the Lagrange Multiplier application just presented there is no explicit function f(r) we are making stationary. On the other hand, one can regard (D.11) that ΣkRk δrk = 0 as the statement δW = 0 where δW = ΣkRk δrk is the total differential "virtual work" done by the constraint forces during a differential virtual displacement {δrk}. Using Newton's law (D.10) this equation can then be written 0 = δW = Σk[mkak - Fk] δrk // D'Alembert's Principle (D.26) in which form it is known as D'Alembert's Principle (1743, Goldstein (1-42)). The analog of f(r) is then an action integral W of the differential virtual work which integral one renders stationary to get δW = 0. The notion of "the action" is mentioned at the end of this appendix. We now present another application of the Lagrange Multiplier technique outlined in (D.25). As before, a certain amount of background is needed before the application can be demonstrated. Generalized coordinates, generalized forces, and Lagrange's Equations Usually the Cartesian components of the particle positions rk are not the most convenient variables to use in solving a problem. Angles are often more useful. With C holonomic constraints one can replace the 3N variables {xn} in (D.17) with a set of 3N-C independent variables {qn} where xn = xn(q1, q2, ....q3N-C,t) n = 1,2..3N (D.27) and one can then rework the above presentation in these new generalized coordinates qj which then won't always have the dimensions of length. For example, one then has, for virtual displacements, δxn = Σj=13N-C(∂xn/∂qj) δqj n = 1,2..3N or δrk = Σj=13N-C(∂rk/∂qj) δqj k = 1,2..N // as in Goldstein (1-44). (D.28) The usual presentation of Lagrangian dynamics with holonomic constraints avoids dealing with the forces of constraint and also avoids the need for using a set of Lagrange Multipliers. That presentation appears in Section V of the Ray and Shamanna and Goldstein p 17 (and many other places) and proceeds like this: 1. Start with ΣkRk δrk = 0 in (D.11) which we write in the form of D'Alembert's Principle (D.26) mentioned above, 0 = Σk=1N [mkak - Fk] δrk (D.26) where Fk is the total non-constraint force acting on particle k. 2. Install (D.28) into D'Alembert's Principle to get 0 = Σk=1N [mkak - Fk] Σj=13N-C()δqj = Σj=13N-C { Σk=1Nmkak - Σk=1N Fk () } δqj = Σj=13N-C { Σk=1Nmkak - Qj } δqj (D.29) where Qj ≡ Σk=1N Fk () . (D.30) The Qj defined above are the generalized forces associated with the "generalized coordinates" qj . 3. Make use of the following non-obvious result (derived in footnote below starting at (D.40)), Σk=1N mkak = () - (D.31) where T is the total kinetic energy of the N particles, T ≡ (1/2)Σk=1Nmk (vk vk) . (D.32) Then (D.29) reads, 0 = Σj=13N-C { () - - Qj } δqj . (D.33) Since the δqj are independent variables, {..} = 0 and we end up with one form of "Lagrange's Equations" () - = Qj j = 1,2....3N-C . // as in Goldstein (1-50) (D.34) If the applied forces can be derived from a potential V(r1, r2, ..... rN) such that Fk = - (k)V(r1, r2, ..... rN) k = 1,2..N // no t or i dependence in V (D.35) then from (D.30), Qj ≡ Σk=1N Fk () = - Σk=1N ( ) () = - j = 1,2....3N-C (D.36) where V(qi) ≡ V(rk(qi)) is a function of the generalized coordinates qi. Since V is not a function of the i we know that = 0. Using this fact, and (D.36) that Qj = – , (D.34) can be written () - = 0 . j = 1,2....3N-C (D.37) The last step is to define the classical Lagrangian L ≡ T - V (D.38) to obtain () - = 0 j = 1,2....3N-C . // as in Goldstein (1-53) (D.39) which is the more conventional form of "Lagrange's Equations" with C holonomic constraints. The equations of (D.39) are usually called the Euler-Lagrange Equations. One can define generalized (canonical, conjugate) momenta pj ≡ so the Lagrange equations (D.39) are just j = or = (q)L . If V = V(qi), then pj = and then if T = Σkmkk2 one has the familiar result pj = mjj. When T = T(i) one finds Qj ≡ - = = j so = Q, which is the generalized Newton's Law in terms of generalized coordinates and generalized forces. Footnote: Derivation of the result (D.31) used above Define the 3N-C independent generalized coordinates qi as in (D.27) according to rk = rk(q1, q2....q3N-C,t) ≡ rk(qi,t) . (D.40) Compute the total time derivative of rk , vk = k = = Σj + = Σj j + . // = vk(qi, i, t) (D.41) Then based on this result compute these two partial derivatives of vk , = Σj j + (D.42) = . (D.43) Next, the total kinetic energy of all k particles is given by (D.32), T ≡ (1/2) Σk mk (vk vk) (D.32) so that = Σk mk vk = Σk mk vk . // using (D.43) (D.44) Apply d/dt to the above, with ak = k : () = Σk mk ak + Σk mk vk () . (D.45) Compute the total time derivative appearing in the second term, () = Σj j + . (D.46) Meanwhile, = Σk mk vk = Σk mk vk [ Σj j + ] // (D.42) = Σk mk vk () . // (D.46) (D.47) Subtract (D.47) from (D.45) to get () - = Σk mk ak (D.48) which is the result quoted above in (D.31). Lagrange Equations with non-holonomic constraints This is our final Lagrange Multiplier example, and it exactly follows the generic prescription of (D.25). The general topic is well described in Goldstein Chapter 2. As the starting point we invoke Hamilton's Principle which is this : δS = 0 where S = !Syntax Error, Idt L(qn(t), n(t), t ) n = 1,2....M (D.49) where there are M independent generalized coordinates qn. The notation is a shorthand to imply that L is a function of all the qn(t) and all the n(t) functions. One might better write L({qn(t)}, {n(t)}, t ) where the {..} indicate sets. L = T - V is the Lagrangian (D.38) with T(i) and V(qi) being the kinetic and potential energies associated with the M generalized coordinates. We shall assume that M = N-D where there were D holonomic constraints on N initial problem coordinates. As shown in (D.60) below ( or see Goldstein p 41 (2-20') based on p 37 (2-15) ) the equation δS = 0 results in the following sum being 0, 1. Σn=1M [ !Syntax Error, Idt { () - } ] δqn(t) = 0 . (D.50) The number 1. on the left refers to the first step outlined in (D.25). In this equation the δqn(t) are M independent functions of t ("path variations"), restricted only by δqn(t1) = δqn(t2) = 0. In order for (D.50) to be true for M arbitrary independent functions δqn(t), one must have {} = 0, () - = 0 n = 1,2....M (D.51) which are the Euler-Lagrange Equations (D.39). Thus it is that the Lagrange Equations can be derived from Hamilton's Principle in the case of only holonomic constraints. But suppose that in addition to the D holonomic constraints there are C non-holonomic constraints, so only M-C of the δqn are independent. In this case the above conclusion (D.51) is incorrect. 2. The C non-holonomic constraints result in C sums Σn Ani δqn = 0 i = 1,2...C analogous to (D.9) above. 3. If λi are C arbitrary factors, certainly Σiλi(ΣnAinδqn) = 0 based on 2. These λi are "Undetermined Lagrange Multipliers". Then !Syntax Error, Idt ( ΣiλiΣnAinδqn ) = 0 as well. 4. Subtract sum 3 from sum 1 to get the new sum Σn=1M [!Syntax Error, Idt { () - - Σi=1C λiAin } ] δqn(t) = 0 . // as in Goldstein (2-26) (D.52) 5. The C λi are selected so that {..} = 0 in the C dependent δqn(t) terms of this sum. The remaining sum has only independent δqn(t) terms so {..} = 0 for all those terms as well. Then all {...} = 0 in the sum of 4. 6. The conclusion is that () - = Σi=1CλiAin . n = 1,2....M // as in Goldstein (2-30) (D.53) This then is the corrected form of the Lagrange Equations in the presence of C non-holonomic constraints. Recalling from (D.7) that Bik = Aik for non-holonomic constraints, the second line of (D.22) with Bin = Ain reads Rn = Σi=1CλiAin . Thus one can interpret the sum on the right of (D.53) as being the force of constraint Rn arising from the C non-holonomic constraints. The forces of the D holonomic constraints are already incorporated into the left side of (D.53) in the reduction of independent coordinates qn from N to M = N-D. The integral S shown in (D.49) is called "the action" and Hamilton's Principle is a classical mechanics incarnation of the Principle of Least Action. In relativistic quantum field theory the action is a spacetime integral of the Lagrangian density L(φn(xμ), ∂μφn(xμ)) where the coordinates qn of L are replaced by fields φn(xμ), and t is replaced by xμ, a point in spacetime. In this case the Lagrange Equations (D.39) take the form (the second line is for comparison), ( ) - = 0 where ∂μ ≡ // Bjorken and Drell (11.30) (D.54) () - = 0 t → xμ qn(t) → φn(xμ) n(t) → ∂μφn(xμ) (D.39) The Case of C holonomic constraints treated as if they were non-holonomic Suppose there are N generalized coordinates qi with C holonomic constraints and 0 non-holonomic constraints. As noted in (D.6) and (D.7), the holonomic constraints can be written in differential form so they look just like non-holonomic constraints. For generalized coordinates qn these two equations would appear as Σn=1N Bindqn + Bitdt = 0 i = 1,2....C // C = s+m or (D.6)' Σn=1N Bin n + Bit = 0 i = 1,2....C where Bin = Ain Bit = Ait for i = 1,2...s non-holonomic Bin = ∂ai/∂qn Bit = ∂tai for i = s+1, s+2....C holonomic . (D.7)' Setting s = 0 we can carry out the procedure of the previous section with Aik replaced by ∂ai/∂qk to obtain this result of rendering the action S stationary, () - = Σi=1Cλi n = 1,2....N (D.55) where the Lagrange multipliers λi are chosen to make this equation be valid for a set of C dependent qn coordinates, as in Step 5 above. Only N-C of the coordinates qn are independent but the equations are written for all N coordinates. Here the functions ai(q1, q2, .....qn, t) = 0 are the C holonomic constraint equations. Footnote: Carry out the δS = 0 functional variation of the action . The variation in each function qn(t) is parameterized in terms of an arbitrary independent function ηn(t) (but which vanishes at both time integration endpoints) and a scalar parameter α : qn(t,α) = qn(t,0) + α ηn(t) δqn(t) = α ηn(t) = ηn(t) ηn(t1) = ηn(t2) = 0 n(t,α) = n(t,0) + α n(t) δn(t) = α n(t = n(t) . (D.56) Then S(α) ≡!Syntax Error, Idt L(qn(t,α), n(t,α), t ) // action (D.49) = !Syntax Error, Idt Σn ( + ) = !Syntax Error, Idt Σn ( ηn + ) . (D.57) But for the second term do a parts integration, !Syntax Error, Idt ( ) = [ ηn(t) ]t2t1 - !Syntax Error, Idt ( ) ηn(t) . (D.58) The "parts" terms vanish since ηn vanishes at both endpoints, so (D.57) becomes, = !Syntax Error, Idt Σn ( - ) ηn(t) (D.59) so δS ≡ dα = { !Syntax Error, Idt Σn ( - ) ηn(t) } dα. (α/α) = { !Syntax Error, Idt Σn ( - ) δqn(t) } dα/α . (D.60) Then, δS = 0 !Syntax Error, Idt Σn [ ( - ) ηn(t) ] = 0 . or (D.61) δS = 0 !Syntax Error, Idt Σn [ ( - ) δqn(t) ] = 0 . But the functions δqn(t,α) = αηn(t) are arbitrary and independent functions of t, so the integral can only vanish if the function in the parentheses vanishes for each term in the sum on n, so δS = 0 ( - ) = 0, n = 1,2...m (D.62) which are the Euler-Lagrange Equations (D.39) and (D.51). For more detail see Goldstein Chapter 2. References The method of Lagrange multipliers often makes cameo appearances in textbooks on the calculus of multiple variables. There are also several pages and pdf's on the web, search on "Lagrange Multipliers". Whole books on the use the Lagrange multipliers for specialized applications may be found at the usual book vendor websites. The links below were last verified Oct 4, 2016. J.D. Bjorken and S.D. Drell, Relativistic Quantum Fields (McGraw-Hill, New York, 1965). R.C. Buck with E.F. Buck, Advanced Calculus, 2nd Ed. (McGraw-Hill, New York, 1965). The discussion of extremum problems with constraints occupies pp 359-363. In the 3rd Edition (McGraw-Hill, New York, 1978), reprinted by (Waveland Press, Long Grove IL, 2003), this discussion has moved to pages 536-540. This excellent and now classic book was first published in 1956. H. Goldstein, Classical Mechanics (Addison-Wesley, Boston, 1950), another classic. There is a 3rd edition 2001 by Goldstein, Safco and Poole, but our page and equation references are to the 1950 edition. E.L. Ince, Ordinary Differential Equations (Longmans, Green & Co. (Ltd.), London, 1927). This classic text became a Dover paperback in 1956 and is available online for $4. S. Jensen, An Introduction to Lagrange Multipliers, http://www.slimy.com/~steuard/teaching/tutorials/Lagrange.html ( includes video lectures) P. Lucht, Tensor Analysis and Curvilinear Coordinates (2016), http://user.xmission.com/~rimrock . If not there, search on . P.M. Morse and H. Feshbach, Methods of Theoretical Physics ( McGraw-Hill, New York, 1953). S. Ray and J. Shamanna, "On Virtual Displacement and Virtual Work in Lagrangian Dynamics", European Journal of Physics, 27 (2006) 311-329. Also https://arxiv.org/abs/physics/0510204v2 . G.E. Shilov, Linear Algebra (Dover Publications, 1977). M.R. Spiegel, Mathematical Handbook of Formulas and Tables ( Schaum's Outline Series, McGraw-Hill, New York, 1968). This is now at 4th Ed (2012) with two coauthors and new reference numbers. W. F. Trench, The Method of Lagrange Multipliers (2013), http://digitalcommons.trinity.edu/mono/7/ , Supplement 2 (31p). This paper also has a "Theorem 1" which is related to but different from ours. Wiki on Lagrange Multipliers: https://en.wikipedia.org/wiki/Lagrange_multiplier M.W. Zemansky, Heat and Thermodynamics 5th Ed (McGraw-Hill, New York, 1968). Our page references are to this edition, but there is a 7th Ed. (1997) with coauthor R.H Dittman and an 8th Ed (2011) in the Special Indian Edition series.