stakgold chap 3 meta meta

Stakgold Chapter 3 Meta Meta Notes PhL 4.22.09 This chapter is p 191-258 or 67 pages of the book. My raw notes are 33 pages, meta notes are 14 pages, and these meta-meta notes are about 10 pages including the Fredholm stuff which Stak omits. Chapter 3: Linear Integral Equations 3.0 The Fredholm Determinant and Related Subjects ( PhL ) 1 3.1 Introduction with Six Examples (191) 3 3.2 The Neumann Series Method (206) 4 3.3 The Spectrum of a self-adjoint H-S operator. (p 212) 4 3.4 The solution of Ku = μu + f for K symmetric and H-S. (220) 6 3.5 Extremal Principles (223) 6 3.6 Approximation Methods Based on Extremal Principles (226) 7 3.7 Continuity and Uniform Convergence: bilinear series and iterated kernels 8 3.8 Approximation Methods for Solving Integral Equations 9 3.9 Non-symmetric HS operators (250) 9 ________________________________________________________________________________ 3.0 The Fredholm Determinant and Related Subjects ( PhL ) Stakgold does not even mention this whole area, but I reviewed it and have lots of notes. The method is only really useful if the λ in our Fred 2 integral equation φ = f + λKφ is "small". In his Chapter 3, Stak does not address the issue of this free parameter λ being small, apart from mentioning the Neumann series. But in QED and scattering perturbation theory it is of great interest, and it showed up a lot in my Geoff Chew hadronic work The basic situation is that we want to solve the integral equation φ = f + λKφ as φ = (1 - λK)-1f = (1 + λK + λ2K2 + ....) f = f + Σm=1λmKm f = ( 1 + Γ ) f where Γ/λ is called the resolvent. This is just the sum of the Neumann series Stak mentions below. If you can find the resolvent, you have basically solved the integral equation. But it is easy to show that, if we put our problem on an N x N lattice, then (1 - λK)-1 = [ cof (1-λK) ]T /det( 1 - λK) = 1 + Γ Now if λ is in some sense "small" (perhaps λ = 1/137), then we can derive power series expansions (in λ) for the two objects [ cof (1-λK) ]T and det( 1 - λK) appearing in the above operator. If we keep only the first few terms in these expansions, then we have an approximate solution for the resolvent. We first proceed as if K were an N x N finite matrix where Kij = k(xi, xj) and we show that det(1-λK) = 1 - (λ) i Kii + (λ2/2)ij - (λ3/3!)ijk + ... and then we take the limit N→∞ to get det(1-λK) = 1 - λ ∫dx k(x,x) + (λ2/2) ∫dx dy - (λ3/3!) ∫dxdydz + .... (goes on forever) and this thing is then called "the Fredholm Determinant". Interestingly, it can also be written entirely in terms of traces of powers of the operator K. det(1-λK) = exp [- Σn=1∞ λn /n tr(Kn)] And there are various other odd theorems floating around. It is also easy to show, going back to the N x N matrix world, that the following is true: [Γij/λ] = [Nij/λ] / det(1-λK) (*) where Nij ≡ [cof (1-λK)]ji – det( 1 - λK) δi,j which is basically our result shown above where Nij is a "minor" (more or less) which appears when you write down the cofactor matrix. [cof (1+B)]ji = (-1)i+j minor(1+B)ji = (-1)i+j detrows≠j;cols≠i When we again take N→∞ we get that [Γ(x,y;λ) /λ] = [N(x,y;λ) /λ]/ det(1-λK) = the resolvent where the fact that Γ and N depend on λ is shown explicitly. A standard notation people use is this: D1(x,y; λ) ≡ [N(x,y;λ) /λ] Fredholm's First Minor (some people erroneously drop /λ) D(λ) ≡ det(1-λK) The Fredholm Determinant Thus, if we know the small λ expansion for the determinant and the first minor, we have at once an expansion in λ for the resolvent, and this in turn solves our integral equation!! The expansion for the minor is this, for finite N and then as N→∞ : [Nij/λ] = Kij – λΣk + (λ2/2!) Σkm – (λ3/3!) Σkmn +. D1(x,y;λ) == k(x,y) – λ∫ds1 + (λ2/2!) ∫ds1ds2 – ... ________________________________________________________________________________ 3.1 Introduction with Six Examples (191) Whereas d/dx is unbounded, many useful integral equation operators are bounded. A subset of such bounded operators are the Hilbert-Schmidt (HS) operators where the double integral of the kernel is finite on the square. Such HS operators are in fact compact = completely continuous and map bounded sets into compact sets, though the significance of this fact is not clear at this point. Stakgold first considers a separable kernel having n terms in the sum with L2 functions. He shows such a kernel is bounded and compact. Taking the limit n→∞ he shows that this compact property is maintained, and he shows that in fact any HS integral operator can be written as such an infinite sum. This then shows that any HS operator is compact. He then mentions the basic kinds of Fredholm Equations: (L-μ)u = 0, homogeneous=EV, and then the inhomo (L-μ)u = f . If μ=0 this latter is called first kind inhomo, otherwise second kind inhomo. The Volterra form with x as an upper integration endpoint can be absorbed into the general fixed-endpoints theory by adjusting the kernel. I directly quote the following note: Note added 4.22.09. I have now read Chapter 1. If we have Lu = μu with some BC's, and if Lu=0 has no non-trivial solutions, we know that Lg=δ has a unique solution g that meets the BC's and then Lu=f with the same BC's has the solution u = Gf where G is the integral operator of kernel g. If L is self-adjoint, then g is symmetric. If we take f = μ1, then Lu = μu becomes u = μGu which is then an integral EV equation for u. The solutions u of this equation respect the BC's. This is why one says that the ODE eigenvalue with BC's can be cast as an integral equation eigenvalue problem which "builds in" the same BC's. Since the integral operator is "nicer" (bounded), maybe best to study the EV problem there instead of as an ODE, at least for certain purposes such as obtaining the general nature of the EV's. Stak then gives six examples with simple kernels, and studies the eigenvalue situation in each case. It is probably mentioned later, but here is an important point. If the kernel k is symmetric, then the EF's of the integral EV problem form a complete set! If the kernel k is a Green's Function g of an ODE where L was self-adjoint, then such a kernel is symmetric. Therefore, the EF's of a self-adjoint L form a complete set! ____________________________________________________________________________________ 3.2 The Neumann Series Method (206) We want to solve our inhomo Fred 2 Ku = μu + F. Divide through by μ to get (1/μ) Ku = u + F/μ. Redefine the last term to be -f, so have (1/μ) Ku = u -f. Set λ ≡ 1/μ so have λKu = u-f. This then says that f = u - λKu = (1-λK)u. If (1-λK)-1 exists, then we can solve to get u = (1-λK)-1f . This type of solution is often used in scattering and is the basis of all of QED, and really all scattering theory, it is pretty major. We can write a formal solution as u = (1 - λK)-1f = (1 + λK + λ2K2 + ....) f 3.25 k2(x,z) = ∫ dz' k(x,z')k(z',z) etc typical term in series: u(x) = ux = λ2 [K2f]x = λ2 ∫dz [K2]xzfz = λ2 ∫dz k2(x,z)f(z) So, to get a solution using this series, you first have to compute all the objects [Kn]xz = kn(x,z) which means doing a bunch of integrals only using the kernel k(x,z), and second you have to integrate each of these iterated kernels against the driving function f(z), then add up all the terms. It is a prescription and it gives a definite and unique solution u. Notice that if λ is an inverse eigenvalue, then Ku = μu λKu = u (1-λK)u=0, where u is an eigenfunction. Now suppose the series for (1-λK)-1 shown above converges. Then if λ is an inverse eigenvalue, we get u = (1-λK)-10 = 0. But this is a contradiction because u is supposed to be an eigenfunction. Thus, the series must not converge if λ is an inverse eigenvalue, or λ EV => not converge. Therefore series converges λ not EV. The series above is the Neumann Series and, when it converges, it is the unique solution. In fact, it always converges when |λ| < 1/||K||, a disc in the λ plane (again, K is bounded we assume). According to the last sentence of the last paragraph, there can be no eigenvalues inside this circle! This translates into saying that Ku = μu must have its eigenvalues inside the disc |μ| ≤ ||K||. This is consistent with our idea later that EV's are bounded by ||K||. Neumann downside: often not easy to compute the Kn. Note that the formal operator (1 - λK)-1 is called the resolvent. (Stak no mention). Example 1 is a good workingman's example, but Example 2 has a special payoff. If your integral equation has the Volterra form, then the Neumann series converges for |λ| < ∞, whereas a normal Fredholm case has |λ| < 1/||K||, so the Volterra Form is very happy with a Neumann series solution. As an application, Stak considers a general initial-value n=2 EV ODE and converts it to a Volterra form integral equation for u"(x). The Neumann series gives a guaranteed solution for u"(x) since Volterra, regardless of value of the EV. Integrate twice meeting BC's and you have u(x). This then proves that the solution to such an ODE exists and its unique. I think this idea could be extended to arbitrary order n, and then this would be a proof of the theorem quoted in Chapter 1 for which we had no proof there. ________________________________________________________________________________ 3.3 The Spectrum of a self-adjoint H-S operator. (p 212) Something missing: Stak never comments on how a H-S operator fits into the general operator theory of Chapter 2. There the spectrum could have 3 types: point, continuous, and that extra thing. I think for self-adjoint, the "extra thing" does not exist. Probably when a and b are both finite (the endpoints), there is no continuous spectrum. In any event, Stak procedes along as if there were only a discrete spectrum for integral equation operators of the type he considers in this Chapter 3. If the kernel k(x,y) is symmetric, then the operator K is self-adjoint (as well as symmetric). In this case the following facts are true: (but the last two items also require that K be HS. ) (1) The eigenvalues of K are all real (2) the EF's of different EV's are orthogonal (3) the multiplicity of any non-zero EV μ is finite. (4) if number of EV's is ∞, then μ=0 is the only possible limit point. I think that part of the magic of symmetric K being HS and therefore compact is this idea that only μ=0 can be a limit point of eigenvalues or, if an eigenvalue, only μ=0 can have an infinite multiplicity. (μ=0 means λ = ∞). At this point we enter a long saga which concerns upper bounds on eigenvalues. It is trivial to show for a bounded K that |μi| ≤ ||K|| (see raw notes). But since the HS K is compact, we get this extra fact: Theorem 6 of page 190 Chap 2: If A is symmetric and completely continuous (compact), then: (1) At least one of these numbers is an eigenvalue: ||A|| or –||A||. (2) There is no eigenvalue μ for which |μ| > ||A||. So compactness of K says we actually hit ||K|| with the largest eigenvalue (in abs value). This then is why we are interested in K being H-S, since H-S implies K is compact. Now, recall that in my notation, || K ||2 = maxu <Ku,Ku>/<u,u> => || K || = maxu ||Ku|| / ||u|| ||| K |||2 = maxu <Ku,u>/<u,u> When the dust settles in my raw notes, we learn using Theorem 6 that for a symmetric HS operator, || K || = ||| K ||| = |μ1| // this is Theorem 1 on page 215 The rest of this section talks about variational methods to "find" (at least approximately) the eigenvalues and their eigenfunctions. Theorem 3A: Subject to <u,φi>=0 for i=1..n, maxu [ <Ku,u>/<u,u>] = |μn+1| and gives φn+1. This suggests an iterative process using an idealized variation method. You first vary u with no conditions to find |μ1| and then, in a perfect world, you know μ1 and φ1. Then you vary u subject to <u,φ1> = 0 and this gives you μ2 and φ2 , and you just keep going. You need some kind of "perfect varying machine" to make this go, and I suspect the computer now provides such a machine! If our varying machine is imperfect, we get some value ak ≤ |μk| because we did not reach the perfect limit. In this case, the thing we get ak is a "lower bound" for |μk|, the thing we are seeking. We now arrive at this strange intermediate theorem (always talking symmetric and HS) Theorem 4: If you take the EF's you get from the above procedure but only include the EF's from non-zero EV's μ≠0, the set of EF's you then have is a complete basis for any function g that can be written g = Kf. In other words, this set of EF's spans the range of K [ f of course is L2 as usual ]. So this explains how you could have a "basis" that might only have a finite number of EF's -- if the range of K is finite dimensional. // This is pretty obvious since the zero-eigenvalue is associated with the nullspace. We then come to the main result: Theorem 5: The full set of EF's of a symmetric HS kernel is complete on L2. Theorem 5A: Obviously, if μ = 0 is not an EV, then the Theorem 4 set of EF's is complete on L2. Theorem 6: The eigenfunctions of any self-adjoint ODE operator L form a complete basis on L2. This is then the justification for writing things like closure/completeness for, say, Legendre functions. But as noted in the raw and meta notes, Theorem 6 depends on a Green's function kernel being H-S, and Stak has never shown that, and it is not obvious to me. This is a loose bolt never resolved. ____________________________________________________________________________________ 3.4 The solution of Ku = μu + f for K symmetric and H-S. (220) Assume that you have found the EV's μi and EF's φi of K. You then want to consider the inhomo2 equation (K-μ)u = f. You know the φi are a complete set, so expand u = Σiaiφi so Ku = ΣiaiKφi = Σiaiμiφi. Also expand f = Σibiφi. Then equation becomes ai(μi- μ) = bi. Stak then systematically considers the four possible cases : μ ≠ 0 and μ ≠ EV; μ ≠ 0 and μ = EV; μ = 0 and μ ≠ EV; μ = 0 and μ = EV; In any case, if μ ≠ EV, you cannot add homo solutions to your particular solution. As usual, the alternative theorem is lurking in all these cases. So basically we have series solutions on the φi for our inhomo2 problem in any of these cases. Convergence is discussed in each case. ____________________________________________________________________________________ 3.5 Extremal Principles (223) Back in Section 3.3 we developed the following two ideas: || K || = ||| K ||| = |μ1| Theorem 3A: Subject to <u,φi>=0 for i=1..n, maxu [ <Ku,u>/<u,u>] = |μn+1| and gives φn+1. In this section, we are going to see what these things look like if we get rid of the absolute values around the eigenvalues. We start by listing the eigenvalues in increasing order, negatives on the left: μ1- μ2- ........ (0) ..... μ2+ μ1+ The above two ideas then translate into the following: ||| K ||| = μ1+ // 3.47 Subject to <u,φ+i>=0 for i=1..n, maxu [ <Ku,u>/<u,u>] = μn+1+ and gives φ+n+1. ` // 3.48 Subject to <u,φ-i>=0 for i=1..n, minu [ <Ku,u>/<u,u>] = μn+1- and gives φ-n+1. // 3.50 When you work on the negative side, the max turns into min as you see. Comment: You can see generally how the above works. If you have a computer scan through the entire L2 function space, your <Ku,u> will eventually hit the max value μ1+ when it finds φ1+. And if you then scan orthogonal to this, the max in that case will be φ2+ which will hit μ2+ , and so on. On the negative side, a scan with no conditions looking to minimize <Ku,u> will hit μ1– when it finds φ1- , and so on. It really is quite simple and reasonable, and of course suggests a search method for finding eigenvalues and eigenfunctions. The trick is to select a smart trial function parameterization. We then have some definitions for qualities of K: +, non-, -, non+, indefinite, real Def: if all eigenvalues are non-negative, then <Ku,u> ≥ 0 and K is said to be non-negative. Def: if all eigenvalues are positive, then <Ku,u> > 0 and K is said to be positive. Def: if all eigenvalues are non-positive, then <Ku,u> ≤ 0 and K is said to be non-positive. Def: if all eigenvalues are negative, then <Ku,u> > 0 and K is said to be negative. Def: Otherwise, the operator K is said to be indefinite. Def: Given u = real, if Ku = real, then K is said to be real. Fact: For integral operator, K is real iff k is real. Courant minimax principle: Since I don't see the significance of this thing, I won't requote it here, see the meta notes. ____________________________________________________________________________________ 3.6 Approximation Methods Based on Extremal Principles (226) The Rayleigh-Ritz Procedure (228) The goal is to try to solve the EV problem Kφi = μiφi. We don't even know how many eigenvalues there are. But we pick n arbitrary but independent functions and call them vi and we make trial eigensolutions u = Σcivi with some real coefficients c. R(u) is called R*. We then vary the ci so as to make stationary R(u) which we know should get us close to μ1. The procedure yields a little nxn matrix equation Ac = 0 where c are our coefficients. There can be a solution if detA = 0, and this is a polynomial of degree n in R*. We then use the largest root R*1 as our resulting estimate (or lower bound) for μ1, knowing R*1 ≤ μ1, and the resulting c as our estimate for φ1 = Σcivi. It is also claimed that R*2 ≤ μ2 and so on, but these lower solutions are far less accurate. Stak then has several remarks: (1) accuracy of μ1 will be much better than accuracy of φ1, due to the stationary idea. (2) accuracy goes down rapidly for lower solutions like μ2, μ3 ...; (3) larger n gives more accuracy; (4) good to select vi based on symmetry. (5) Things are simpler if you use orthonormal vi, but you don't have to. (6) if you could use n = ∞, you would get exact results since your vi would be a complete basis, but of course this is not practical. Stak then gives a single good example of using the R-R procedure on page 231. In the nuts and bolts of R-R, you first "precompute" the integrals Kik = <Kvi, vk> and αik= <vi, vk>. For some complicated K, this can only mean numerical integration, so we are talking computer work here, not analytic stuff. Then you compute the n x n determinant shown in 3.55, and then you have to find the roots of your polynomial. If you pick orthonormals, then αik = δik and the determinant is the much simpler one shown in 3.57. Matrix K will be symmetric. Basically, we are making an n x n matrix approximation to our ∞ x ∞ matrix problem. The Schwarz Iteration Procedure (p 231) // abbreviated quote from meta notes Again, the goal is to solve the EV problem Kφi = μiφi. Recall that we used iterated kernels Kn in the Neumann series method for solving the inhomo equation written as u = f + λKu and we found that the series was u = (1 - λK)-1f = (1 + λK + λ2K2 + ....) f. In the Schwarz method, we again make use of these iterated kernels Kn, but they are used differently and of course for the homo equation. The idea is to start off with some seed function f0 and then compute fn = Knf0. If you are lucky, this sequence will become stable after a while and you get fn+1 ≈ α fn which says fn+1 = Kfn ≈ αfn and thus this stabilized limiting function fn is an eigenfunction (or an approximation for one), eigenvalue α. That is the first idea. The second part of Schwarz is that you define an ≡ <fo,Knf0> and consider the sequence for large n. You define the ratio θn = an+1/an (Schwarz quotient) . Basically the claim is that lim θn ≤ μ1. Upper Bounds to Eigenvalues Our usual variation methods like R(u) = <Ku,u>/<u,u> always give lower bounds for eigenvalues, it would be nice to have some upper bounds so you could bracket (enclose) the eigenvalue to know how accurate you might be. The ideas here are based on the notion that if you define σ ≡ <Kv,v > α2 ≡ || Kv - σv ||2 = ||Kv||2- σ2 if v is an exact normalized eigenfunction, then α = 0 and σ = μ. If v is only an approximate normalized eigenvalue, then you find by the WBFS Theorem (four names, no web hits so no date) that your eigenvalue is bracketed in this way, reminding me of a mean and standard deviation idea, σ - α ≤ μ ≤ σ + α ___________________________________________________________________________________ 3.7 Continuity and Uniform Convergence: bilinear series and iterated kernels Uniform convergence is more demanding that convergence in the mean, and this section shows that certain partial sums of functions of x and ξ converge uniformly "on the square" to the infinite sum limits. Preamble: Consider self-adjoint ODE Lu=f with no nullspace, so solution is u = Gf. Note that u is "in the range of integral operator G" and therefore by a theorem above we know we can expand u on the complete set φi(x), since G is symmetric. So can write u(x) = Σiaiφi(x). Similarly, can write f(x,ξ) = Σi Ciφi(x)i(ξ) which is called a bilinear expansion of a function of two variables. Stak then shows: Theorem 4: kn(x,ξ) = Σiμin φi(x) i(ξ), and this converges uniformly on the square, n ≥ 2. Mercer's Theorem: For suitable extra conditions on K, the above is also true for n = 1. If we assume these "extra conditions", then we obtain this somewhat impressive result: g(x,ξ) = Σi∞μiφi(x)i(ξ) or RHS = Σi∞ <x|i>μi<i|ξ> = Σi∞ <x| μi| i> <i|ξ> = Σi∞ <x| G | i> <i|ξ> = <x|G|ξ> = g(x,ξ) which gives the Green's function of Lg=δ in terms of the φi of Gφi = μiφi. This result did not appear anywhere in Chapter 1 on "Green's Functions". Page 72 (1.77) looks a little like it, but not really because there are no eigenfunctions involved in (1.77). Theorem 4 (or Mercer's) says that the sum shown equals the LHS, and the partial sum to some N converges as N→∞ to the LHS uniformly on the square. If you use "in the mean" convergence, Theorem 4 applies for n ≥ 1. Just integrate Theorem 4 over x and use orthog of φi to get Theorem 4A: ∫dx kn(x,x) = Σiμin // an interesting Parseval like sum rule n ≥ 2. which follows from Theorem 4 since kn(x,x) = Σiμin φi(x) i(x) and the φi are normalized. In our QM notation we can write this as RHS = Σi∞ <i |μin|i> = Σi∞ <i |Kn|i> = trace(Kn) = ∫dx <x|Kn|x> with the idea that trace is independent of basis. Theorem 4B: [∫dx kn(x,x)]1/n ≥ μ1 if K is non-negative, an upper bound for μ1 n ≥ 2 But is obvious from 4A since 4A says: ∫dx kn(x,x) = μ1n + Σi≠1μin ≥ μ1n. Note: Lg=δ goes with Lu=f which goes with u = Gf which we usually write as Gu = f as in 3.4 p 195, and this is the inhomo1 problem where parameter μ = 0. We can write our n=1 case above as: (μi = 1/λi) g(x,ξ) = Σi∞ φi(x)i(ξ)/λi and this Green's Function is associated with the parameter μ=0 (λ=∞) situation. Later in Chap 4 we shall learn the more general result for the inhomo2 problem with parameter μ = 1/λ which is this: g(x,ξ;λ) = Σi∞φi(x)i(ξ)/(λi-λ) // p 273 (4.42) where this g is defined by (L - λ)g = δ as in 4.1 page 259. ____________________________________________________________________________________ 3.8 Approximation Methods for Solving Integral Equations For gory details see Kantorovich and Krylov 1958 English translation. In this section we discuss 5 different plans for finding approximate solutions to our inhomo2 integral equation. 1. Approximate area as little strips, then do lattice integral (Trap and Simpson versions allowed). 2. Approximate the kernel k as a finite separable sum of independent functions. 3. For some {vi} approx the solution u(x) = Σicivi(x), install in the IE, then do one of two things: (a) minimize the L2 error of the two sides of the IE (Best in the Mean); (b) require that both sides have the same projection on some finite subspace spanned by some wi (Galerkin's Method). 4. Two variational methods are presented, theory only: maximize either I(u) or J(u). 5. Carry out the maximize I(u) plan, get the Rayleigh-Ritz Equations, same as a Galerkin case. All of these methods reduce the problem of solving Ku=f to an nxn linear algebra problem. See meta notes for a little summary of each of the above items. Exercise 3.23. Discusses the extended (generalized) eigenvalue problem Nu = λMu. __________________________________________________________________________________ 3.9 Non-symmetric HS operators (250) For non-symmetric K, it turns out that you can still do pretty much in terms of solving an integral equation of the form Ky = h, but you cannot make use of the φi EF's of K because these are not in general a complete set, as a very simple example shows. However, the EF's vn and un of the left and right iterated kernels KL = KK* and KR= K*K can be used instead! Operators KL,R are both HS (if K is), are symmetric, have the same eigenvalues, and are non-negative, so they have all the properties we like! For example, either set un or vn is complete. The upshot is that you can solve Ky = h as long as h meets a few restrictions. Along the way, we learn that the vn span the perp space of K, while the un the perp space of K*. These facts lead to a final solution in the usual form we expect such as y = y0 + Σi [(h.ui)/μi] vi . Notice that both the ui and vi functions are involved in this formula which solves Ky=h. Everything studied in this section is illustrated in the simple Example which ends the section.