prob theory Ng and spectral

Probability theory Ng and spectral PhL 3.26.05 I think this is now cleared up and Spectral Doc now has Appendix D. 1. Let's try to establish the analogy between Ng trials and pulse trains. In each "trial" a set of "features" are measured. Movie cuddly or not, action or not, etc. In the pulse train, cuddly/non-cuddly = ym and action/no_action = yn where ym and yn are the amplitudes of two cars in the train. The number of trials goes with the number of pulse trains in the ensemble and that has index i and total count I. The analogy is not quite right because ym takes values from the same set as yn. 2. Consider this expectation value from my Ng notes: E(XY) = !Syntax Error, I!Syntax Error, I dx dy x y p(x,y) Here each variable takes a continuum of values in (-∞.∞). The integration endpoints could be (a,b) for some other case. But I am interested in the case where X can take certain discrete values we shall denote by index k, not to be confused with i. So we have yk and xk and each has perhaps a separate set of discrete values. Then the above would become E(XY) = ΣkΣk' xk yk' p(xk,yk') Here p(xk,yk') is the probability in the entire ensemble that X takes the value xk at the same time that Y takes the value yk'. If X and Y are correlated (features or pulse train positions), we cannot factor p into a simple product of two functions. If things are uncorrelated, we factor in this way p(xk,yk') = pX(xk)pY(yk') and then E(XY) = ΣkΣk' xk yk' p(xk,yk') = E(X)E(Y) 3. Now consider this claim: E(XY) = (1/I) Σi=1I x(i)y(i) Two questions: Is this true? How is it related to item 2 above? Here x(i) is the value x takes in the ith trial and y(i) is the value y takes in that same trial. How does this connect with pulse trains? E(YmYn) = (1/I) Σi=1I ym(i)yn(i) perhaps m-n = k Now the subscripts m and n are not summation indices, they are just labels which distinguish what I called X and Y in the previous equation. These are static labels for the moment. If we sum over all pairs as shown over the ensemble of trials or pulse trains, that is going to be the average value of that quantity. In this form for E(XY), there is no probability function appearing as in the previous form. Instead, we are using the actual values from experiment, and that experiment knows about the probability function! The expectation value of some quantity must be the average value of that quantity if you average over a billion trials or pulse trains in an ensemble. So I think the answer to the first question is: yes, it is true. 4. If you had the experimental data from item 3, is there some way to compute the p function of item 2? I suspect you would need all "moments" and not just this one moment. But it must be possible to go the other direction, and I have already shown how, given p, you can compute E(XY). 5. In the pulse train case, I guess I am happy then with these two equations for non-correlated cases. E(ymyn) = E(ym)E(yn) E(ym) = [p] A + (1-p) B 6. What about the standard deviation stuff? We can write E([X-E(X)]2) = Σk [xk - E(X)]2 pk(xk) or E([X-μx]2) = Σk [xk - μx]2 pk(xk) = Σk xk2 pk(xk) + μk2 Σk pk(xk) - Σk (2μxxk) pk(xk) = E(X2) + μk2 - 2μx2 = E(X2) - E(X)2 = variance = σ2 7. OK, I think all is well. What is correlation and covariance then? corr(X,Y) = E(XY) = ΣkΣk' xk yk' p(xk,yk') // wrong!!! I think this might be wrong! Ouch, I will then have an Ng mistake. Luckily I only made a small Ng comment about it. My Proakis notes show no corr(X,Y) object. Where did I get that idea from? Here is a wiki note which shows me wrong. Here is another PDF source confirming the above, So I need to fix this in my Ng notes very soon! There is only one occurrence of corr( in that doc so that will be easy to fix. // Ng notes are now all fixed and pushed out, so we can continue here. That was a very ugly error to have in t public document. 8. I am now ready to rewrite a small part of spectral doc relating to the above subject. // Well, it ended up being a whole new Appendix D.