old App D on probability stuff

Appendix D: How α and β are related to μ and σ We first review some of the basics of random variables X and Y: expected values, means, covariance, variance and standard deviation. We do this in a context in which each variable takes on a set of discrete values rather than continuous values. We then translate our various facts into the language of X,Y → Ym Yn where the latter are random variables associated with the amplitudes at positions m and n of a pulse train. The labels m and n add a notational complexity that is a little difficult to comprehend unless things are written out explicitly, which is done below. Appendix G contains "the long version" of this appendix. Here, we quietly avoid the question of "What is a random variable ?", and "Why are positions in a pulse train statistically equivalent ?" . These and related questions are addressed in some detail in Appendix G to which the reader is referred. Here we are mainly interested in relating our pulse train parameters α and β to statistical quantities μ and σ in the case of an uncorrelated pulse train. (a) Random Variables X and Y If X and Y are random variables which can take discrete values xk and yk, one can express the mean value of each variable as μx = E(X) = Σk xk px(xk) μy = E(Y) = Σk yk py(yk) (D.1a) where E(X) means the expected value of X. The sum Σk is over all values in the set of values that X or Y can take. Probability px(xk) is the probability that X takes the value xk and similarly for py(yk). If we have a very large ensemble of I systems we can measure the above quantities in this manner, μx = E(X) = (1/I) Σi=1I x(i) μy = E(Y) = (1/I) Σi=1I y(i) (D.1b) where i labels a particular system in the ensemble, and x(i) and y(i) are the values X and Y take in ensemble system i. The probability functions like px(xk) do not appear in (D.1b), but they are present in spirit, being embedded in the "experimental data" like x(i). The expected value E(XY) is defined by E(XY) = ΣkΣj xk yj pxy(xk,yj) (D.2a) where now pxy(xk,yj) is the joint probability that X has value xk when Y has the value yj. One would measure E(XY) in this manner, E(XY) = (1/I) Σi=1I x(i) y(i) . (D.2b) If the random variables X and Y are independent, then pxy(xk,yj) = px(xk) py(yj) // independent (D.3) and then it is obvious from the above definitions of E(XY), E(X) and E(Y) that E(XY) = E(X)E(Y) . // independent (D.4) The covariance of X and Y is defined as, cov(X,Y) = E( [X - μx] [Y - μy] ) = ΣkΣj (xk - μx) (yj- μy) pxy(xk,yj) . (D.5a) It could be measured for an ensemble in this way cov(X,Y) = (1/I) Σi=1I (x(i) - μx) (y(i)- μy) (D.5b) where μx and μy would be the measured values of E(X) and E(Y) as shown in (D.1b). Note that, cov(X,Y) = E( [X - μx] [Y - μy]) = E(XY) - μxE(Y) - μyE(X) + μxμyE(1) = E(XY) - μxμy - μyμx + μxμy = E(XY) - μxμy . (D.6) If X and Y are independent random variables then cov(X,Y) = E(XY) - μxμy = E(X)E(Y) - μxμy = μxμy- μxμy = 0 . // independent (D.7) In the special case that X and Y are the same variable, we write cov(X,X) = var(X), the variance of X, which is also defined as the square of the standard deviation σx, var(X) ≡ σx2 ≡ cov(X,X) = E(X2) - μx2 (D.8a) with experimental measurement from (D.5b) as var(X) ≡ σx2 ≡ cov(X,X) = (1/I) Σi=1I (x(i) - μx) 2 . (D8.b) Finally, the correlation of X and Y is defined as corr(X,Y) = cov(X,Y) / [σ(X)σ(Y)] (D.9) and we get the result from (D.7) that if X and Y are independent, then corr(X,Y) = 0 and the two variables are thus uncorrelated. (b) Random Variables Ym and Yn We now take a special case of X and Y as follows: X = Ym = random variable associated with the amplitudes ym(i) at pulse location m in pulse train Y = Yn = random variable associated with the amplitudes yn(i) at pulse location n in pulse train Things are a little complicated due to the extra labels m and n, but we can nevertheless carefully translate all the X and Y equations above. Also, we shall add the following extra notation for an expected value to be consistent with our earlier work, <x> ≡ E(X) <xy> ≡ E(XY) <x2> ≡ E(X2) . Here then are all the equations of the previous section translated from X,Y to Ym,Yn . We have primed the equation numbers of the previous section. μYm = E(Ym) = Σk ym(k) pYm(ym(k)) = <ym> μYn = E(Yn) = Σk yn(k) pYn(yn(k)) = <yn> (D.1a)' μYm = E(Ym) = (1/I) Σi=1I ym(i) = <ym> μYn = E(Yn) = (1/I) Σi=1I yn(i) = <yn> (D.1b)' E(YmYn) = ΣkΣj ym(k) yn(j) pYmYn(ym(k), yn(j)) = <ymyn> (D.2a)' E(YmYn) = (1/I) Σi=1I ym(i) yn(i) = <ymyn> (D.2b)' pYmYn(ym(k), yn(j)) = pYm(ym(k)) pYn(yn(j)) // independent (D.3)' E(YmYn) = E(Ym) E(Yn) or <ymyn> = <ym><yn> // independent (D.4)' cov(Ym, Yn) = E( [Ym-μYm] [Yn-μYn] ) = ΣkΣj (ym(k)- μYm) (yn(j)- μYn) pYmYn(ym(k), yn(j)) (D.5a)' cov(Ym, Yn) = (1/I) Σi=1I (ym(i) - μYm) (yn(i)- μYn) (D.5b)' cov(Ym, Yn) = E(YmYn) - μYm μYn = <ymyn> - μYm μYn (D.6)' cov(Ym, Yn) = 0 . // independent (D.7)' var(Ym) ≡ σYm2 ≡ cov(Ym, Ym) = E(Ym2) - μYm2 = <ym2 > - μYm2 (D.8a)' var(Ym) ≡ σYm2 ≡ cov(Ym, Ym) = (1/I) Σi=1I (ym(i) - μYm) 2 (D8.b)' corr(Ym, Yn) = cov(Ym, Yn) / [σ(Ym)σ(Yn)] (D.9)' (c) Statistical Pulse Trains The arm-waving argument is now made that for a large ensemble of statistical pulse trains, there should be no distinction between different positions (like m and n) in the pulse train. This argument seems particularly compelling if the pulse trains of the ensemble are each infinitely long. This is really just an intuitive argument, and the notion is supported quantitatively by the "useful ensemble" discussion in Appendix G (g), and we shall refer to such an ensemble of pulse trains as being a "statistical ensemble". With no further comment here, we shall accept this fact that no position is different from any other, and this leads to a simplification of the equations above. In particular, μYm = μYn ≡ μ, σYm = σYn ≡ σ pYm(z) = pYn(z) = p(z) . (D.10) One other result ( as shown in Appendix G), is this fact, written two ways <ymyn> = depends only on m-n <ymym+k> = depends only on k . (D.11) With these simplifications, we write our equation set a third time, now with double-primed numbers, μ = E(Ym) = Σk ym(k) p(ym(k)) = <ym> μ = E(Yn) = Σk yn(k) p(yn(k)) = <yn> (D.1a)" μ = E(Ym) = (1/I) Σi=1I ym(i) = <ym> μ = E(Yn) = (1/I) Σi=1I yn(i) = <yn> (D.1b)" E(YmYn) = ΣkΣj ym(k) yn(j) pYmYn(ym(k), yn(j)) = <ymyn> (D.2a)" E(YmYn) = (1/I) Σi=1I ym(i) yn(i) = <ymyn> (D.2b)" pYmYn(ym(k), yn(j)) = p(ym(k)) p(yn(j)) // independent (D.3)" E(YmYn) = E(Ym) E(Yn) or <ymyn> = <ym><yn> // independent (D.4)" cov(Ym, Yn) = E( [Ym-μ] [Yn-μ] ) = ΣkΣj (ym(k)- μ) (yn(j)- μ) pYmYn(ym(k), yn(j)) (D.5a)" cov(Ym, Yn) = (1/I) Σi=1I (ym(i) - μ) (yn(i)- μ) (D.5b)" cov(Ym, Yn) = E(YmYn) - μ2 = <ymyn> - μ2 (D.6)" cov(Ym, Yn) = 0 . // independent (D.7)" var(Ym) ≡ σ2 ≡ cov(Ym, Ym) = E(Ym2) - μ2 = <ym2 > - μ2 (D.8a)" var(Ym) ≡ σ2 ≡ cov(Ym, Ym) = (1/I) Σi=1I (ym(i) - μ) 2 (D8.b)" corr(Ym, Yn) = cov(Ym, Yn) / [σ2] (D.9)" Of special interest are these equations taken from the above list : <ym> = (1/I) Σi=1I ym(i) = μ no dependence on n (D.1a)" <ymyn> = (1/I) Σi=1I ym(i) yn(i) depends on m-n (D.2b)" <ym2> = (1/I) Σi=1I [ym(i]]2 = σ2 + μ2 no dependence on n (D.8a)" Since <ym2> = σ2 + μ2 is independent of m, we can give it the name β so that β = <ym2> = σ2 + μ2 . (D.12) For certain kinds of pulse trains, <ymyn> does not depend on m-n and is in fact then independent of both indices m and n. In this case we define the constant α = <ymyn> for m≠n . So we then have α = <ymyn> for m≠n // for certain kinds of pulse trains of our statistical type β = <ym2> = σ2 + μ2 // for any pulse train of our statistical type (D.13) One of those special kinds of pulse trains is one in which Yn and Ym are independent random variables (hence uncorrelated) for m ≠ n. In this case we find α = <ymyn> = <ym><yn> = μ * μ = μ2 // independent (D.14) and in this case we then have μ ≡ <ym> // general case α = <ymyn> = μ2 // independent only β = σ2 + μ2 . // general case (D.15) We can now restate equation (35.6) in this more statistically-oriented manner, <ym> = μ = [p]A + [1-p]B n ≠ m α ≡ <ymyn> = μ2 = { [p]A + [1-p]B }2 = [pp] AA + [p(1-p)] 2AB + [(1-p)(1-p)]BB n = m β ≡ <ym2> = [p]AA + [(1-p)] BB = σ2 + μ2 (D.16) which is to say α = μ2 β = μ2 + σ2 = α + σ2 σ2 = (β-α) . (D.17) Using the above expressions for α and β we can have Maple compute σ2: which says σ2 = variance = [ p(1-p)] (A-B)2 . (D.18) If p = 0 or p = 1, this variance vanishes as one would expect.