Section 35

35. Statistical Pulse Trains What is a Statistical Pulse Train? Imagine a very long sequence whose elements are symbols in some set. Perhaps this sequence is a billion symbols long. The sequence elements can be thought of as amplitudes of a pulse train (a signal) which is marching along in time left to right. The sequence elements are labeled xi for i = -I ...+I where I is very large, and temporally the first sequence element is x-I and the last is xI. The sequence can be written [ x-I, x-I+1....... x-2, x-1, x0, x1, x2......xI ] We run the sequence through a machine that looks like this: The sequence values xi are represented by the dots which move left to right in time. Each box can hold one sequence value at a time. The boxes, a set of toll booths in series, cooperatively calculate certain statistical quantities of interest. Each box Yn separately makes this calculation based on the symbols that pass through, E(Yn) = (1/K) Σi=-IK xi . This is the mean of the xi after K symbols have passed through. Another calculation is performed by a cooperative effort of the Y0 and Y2 boxes: E(Y0Y2) = (1/K) Σi=-IK xixi+2 . Box 1 also takes note of the passing xi values and creates a distribution for the passing symbols by just binning values in bins and dividing on the fly by the number of symbols so far passed by, pY1(x) = (1/K) Σi=1K ( xi = x ) for all x values in the symbol set . Meanwhile, box 0 and 2 compute this joint distribution function by a similar binning method, pY0Y2(x,y) = (1/K) Σi=-IK ( [ xi, xi+1] = [x, y] ) N boxes cooperate on this Nth order joint distribution calculation pY1Y2Y3...YN(x,y,z...) = (1/K) Σi=-IK ( [ xi,xi+1, ...xi+N] = [x,y,z.....] ) . When the long sequence has finally passed through all the boxes, we have accumulated lots of data. We would like to associate with box Yn a random variable we shall call Yn. The values yi this variable takes are the values over time of the symbols xi which momentarily sat in its work station position as the sequence passed through. These values, which we assume are real numbers, have a distribution function pYn(x), so indeed, Yn is a "random variable" (see Appendix G). One thing is very clear from the above picture. After the entire sequence has passed through, E(Yn) is the same for all n ≡ <yn> E(YnYn+s) is the same for all n ≡ <ynyn+s> (35.1) where we introduce the notation <...> to mean an expected value. The simple reason is that each box or separated pair of boxes sees the exact same stream of symbols pass by. In the field of statistics, a set of rules like (35.1) is associated with the word "stationary" -- our little toll booths are associated with "stationary stochastic processes". When the entire sequence has passed through all the boxes, we have for example this result which does not depend on index n (but in general does depend on s), E(YnYn+s) = Σi=-II xixi+s . For each value of s, this makes use of a different pair of boxes (random variables). If the data is assembled for a set of s values, the result is the autocorrelation sequence as defined above in (32.16) or (32.16)' for a finite sequence, rs = E(YnYn+s) = (1/I) Σi=-II xixi+s ≡ <ynyn+s> If the random variables Yn and Yn+s are independent, then ( see Appendix G ) for s ≠ 0, E(YnYn+s) = E(Yn) E(Yn+s) or <ynyn+s> = <yn><yn+s> = [<yn>]2 and in this case <ynyn+s> depends on neither n nor s. Rather than being characterized by the specific symbols xi, we can instead characterize the pulse train by its complete set of statistical properties, such as the expected values shown above. In a distant way it is like describing a sequence xi by is FFT coefficients, or an analog function x(t) by its spectrum X(ω). We shall refer to a pulse train (or sequence) so characterized as being a "statistical pulse train" or a "statistical sequence", though the terms "random pulse train" and "random sequence" are more common. As discussed in Appendix G, there is a certain incorrect impression sometimes assumed when the word "random" is used to describe something. For example, if a sequence of 0's and 1's has mean <yn>= 1/3, that does not preclude it from being a random sequence. (a) Spectral Power Density for a Statistical Pulse Train We call upon our expression (34.14a) which says P(ω) = Ppulse(ω) | Y"(z) |2 = Ppulse(ω) R"(z) (34.14a) where R"(z) is the Z transform of the autocorrelation sequence rs = <ynyn+s> , R"(z) = !Syntax Error, I rs z-s = !Syntax Error, I <ynyn+s> z-s For any given pulse train type, our task then is to compute <ynyn+s>. If we assume that our random variables Yn are independent, this task simplifies greatly, since in that case rs = <ynyn+s> = <yn>2 = μ2 for s ≠ 0 r0 = <yn2> where we refer to the mean of the sequence symbols as μ and have introduced two constants α and β as shown. In this case the autocorrelation sequence has this frighteningly simple form, The variance (standard deviation squared) for our distribution is easily computed (App G) as σ2 = E(Yn2) - E(Yn)2 = <yn2> - <yn>2 = <yn2> - μ2 and then it is convenient to express things in terms of μ and σ <ymyn> = μ2 // Yn are independent <ym> = μ <yn2> = σ2 + μ2 We are now ready to compute R"(z), where we now assume an infinitely long pulse train, R"(z) = !Syntax Error, I <ynyn+s> z-s = !Syntax Error, I{ μ2 } z-s + { σ2+μ2 } = μ2 !Syntax Error, I z-s + σ2 Since z = eiωT from (24.1) we have z-s = e-iωTs . Then we can use (13.2) (valid for either exponential sign) !Syntax Error, Ie-ink = !Syntax Error, I2πδ(k - 2πm) -∞ < k < ∞ . (13.2) with k = ωT1 to get !Syntax Error, I z-s = !Syntax Error, I2πδ(ωT1 - 2πm) and we thus have, R"(z) = σ2 + μ2 !Syntax Error, I2πδ(ωT1 - 2πm) so the spectral power density is then P(ω) = Ppulse(ω) R"(z) = Ppulse(ω) [σ2 + μ2 !Syntax Error, I2πδ(ωT1 - 2πm)] Since 2πδ(ωT1 - 2πm) = (2π/T1) δ (ω - 2πm/T1) = ω1 δ(ω - mω1) we have the alternate forms P(ω) = Ppulse(ω) R"(z) = Ppulse(ω) [σ2 + ω1 μ2 !Syntax Error, I δ(ω - mω1) ] = σ2 Ppulse(ω) + ω1 μ2 !Syntax Error, I Ppulse(mω1)δ(ω - mω1) In general, the spectrum has a continuous part σ2Ppulse(ω) and the rest is discrete. Some of the spectral lines may be killed off if Ppulse(mω1) = 0 for certain values of m. If the symbol mean μ vanishes, the discrete part goes away. The continuous part is in effect created by the variance of the distribution pYn(x). Verification To find external verification for ***, we write things in terms of f where ω = 2πf, Ppulse(ω) = = // (34.4) and text after (1.4) P(f) = 2πP(ω) // (34.4) last item 2π δ(ωT1-2πm) = 2π δ(2πfT1-2πm) = (1/T1) δ( f - m/T1) . Then *** becomes <P(f)> = { σ2 + !Syntax Error, I δ( f - m/T1) } or <P(f)> = { σ2 + !Syntax Error, I δ( f - m/T1) } (35.11b) This may be compared to result (A.17) in Appendix A of Xiong which we quote, If the pulse train has a finite length with yi index running from -N to N, we backtrack and make adjustments: resume here, I think the sum endpoints are wrong R"(z) = !Syntax Error, I <ynyn+s> z-s = !Syntax Error, I{ μ2 } z-s + { σ2+μ2 } = μ2 !Syntax Error, I z-s + σ2 Since z = eiωT from (24.1) we have z-s = e-iωTs . Then we can use (13.3) (valid for either exponential sign) !Syntax Error, I eink = 2π δ5(k,N) (A.30) with k = ωT1 to get !Syntax Error, I z-s = 2π { } ≡ 2π δ5(k,2N) and we thus have, R"(z) = σ2 + μ2 2π δ5(k,2N) = σ2 + μ2 2π { } and therefore P(ω) = Ppulse(ω) [ σ2 + μ2 2π { } ] which is a completely continuous spectrum. For large N the δ5 function has strong but continuous peaks equally spaced, and as N→∞ these peaks become delta functions, limN→∞ δ5(k,2N) = limN→∞ = !Syntax Error, Iδ(k-2πm) (A.19) and we then obtain P(ω) = Ppulse(ω) [ σ2 + μ2 2π {!Syntax Error, Iδ(k-2πm) } ] which replicates our earlier result for the infinite pulse train. (b) Symbols in the set {A,B} ______________________________________________________ As a simple example, suppose the set of symbols that our yi can take is just {A,B}. Then the pmf distribution associated with random variable Yn (see Appendix G) is quite simple pYn(x) = p if x = A pYn(x) = 1-p if x = B . We can then compute μ = <ym> = [p]A + [1-p]B For n ≠ m we then have <ymyn> = <ym>2 = μ2 = { pA + (1-p)B }2 = [pp] AA + [p(1-p)] AB + [(1-p)p]BA + [(1-p)(1-p)]BB and for n = m, <ym2> = [p]AA + [(1-p)] BB . In the square brackets [..] we indicate the probability of some case occurring, and this is multiplied by the value that the quantity in question takes in that case. Notice how the long expression for <ymyn> makes complete sense if one enumerates all possibilities. The variance may be computed as σ2 = <yn2> - μ2 = [p]AA + [(1-p)] BB - { pA + (1-p)B }2 = [p(1-p)] (A-B)2 where Maple helps out Thus, for a statistical pulse train with independent amplitudes in the set {A,B} we find then that P(ω) = Ppulse(ω) [σ2 + μ2 !Syntax Error, I2πδ(ωT1 - 2πm)] = Ppulse(ω) [ p(1-p)(A-B)2 + (pA + (1-p)B)2 !Syntax Error, I2πδ(ωT1 - 2πm)] For the interesting case that p = 1/2 this becomes P(ω) = Ppulse(ω) [ (A-B)2 + (A+B)2 !Syntax Error, I2πδ(ωT1 - 2πm)] (d) Statistical Non-Correlated Pulse Trains: Summary and Examples Here then is a brief summary of the above results: Spectral Power Density of a Non-Correlated Statistical Pulse Train (xxx) x(t) = !Syntax Error, I yn xpulse(t - nT1) // or !Syntax Error, I for a finite pulse train P(ω) = Ppulse(ω) [σ2 + μ2 !Syntax Error, I2πδ(ωT1 - 2πm)] // infinite (xxx1) P(ω) = Ppulse(ω) [ σ2 + μ2 2π { } ] // finite (xxx) If symbols are restricted to {A,B} then σ2 = p(1-p) (A-B)2 and for p = 1/2 σ2 = (A-B)2 μ2 = (pA + (1-p)B)2 and for p = 1/2 μ2 = (A+B)2 Example: Suppose A = 1 and B = 0 so that σ2 = p(1-p) and μ2 = p2. If we go on to assume p = 1, then every pulse has amplitude 1 and we have a simple pulse train. In this case σ2 = 0 and μ2 = 1 so P(ω) ≡ Ppulse(ω)!Syntax Error, I 2πδ(ωT1 - 2πm) joules which agrees with our earlier simple pulse train result ****. As we reduce p below p = 1, the line spectra are scaled down by μ2 = p2 < 1, and a continuous spectrum starts to appear with σ2 = p(1-p). The randomness (variance σ2) of the amplitudes creates a continuous component in the spectral power density. As p reaches p = 1/2, we get σ2 = 1/4 and μ2 = 1/4 to give this classic result P(ω) = Ppulse(ω) [(1/4) + (1/4) !Syntax Error, I2π δ(ωT1- 2πm) ] Finally, when p reaches p = 0, then every pulse has B = 0, x(t) ≡ 0, σ2 = 0, μ2 = 0, and so P(ω) = 0. There is nothing there! ok to here (e) A numerical example of a Statistical Pulse Train Recall from box (34.4) that for a finite pulse train, P(ω) = Ppulse(ω) = T = (2N+1)T1. (35.20) Therefore we can write our p = 1/2 Example 3 result (35.19) as = |Xpulse(ω)|2 [(1/4) + (1/4) 2π δ6(ωT1,N) ] . (35.21) Question: Does my δ5 result really differ from this δ6 result? I expected them to be exactly the same in full detail! This might indicate an error somewhere. δ5(k,2N) = δ6(k,N) = Why are these so different?? I plotted them, they are definitely not the same. Something is amiss! δ6(k,N) = We shall use for xpulse(t) a square pulse of height A=1 and width τ = T1 = 1 so that, from (9.2), |Xpulse (ω) | = sinc(ω/2) . (35.22) Since our pulse train will be fairly short (N = 20 pulses) and since we shall only average a small number of pulse trains (M = 10), we know our result will not exactly match (35.21). Still, we hope to see in our result some kind of continuous background spectrum which approximates the curve (1/4)|Xpulse(ω)|2 = (1/4) sinc2(ω/2), and we expect to see delta-function-like peaks which, since 2πδ6(0,N) = (2N+1), have a peak value of about (1/4)41 = 10.25. Since this will be added to the continuous background, the peak should have a height of 10.25 + .25 = 10.5. However, for our small ensemble, we won't have exactly p = 1/2, so the delta peak won't be exactly 10.5 units high. We know that δ6 has identical peaks spaced by 2π, but we expect the non-central peaks to be suppressed by the sinc2(ω/2) zeros which occur at ω = n(2π). Here is the self-documented Maple program which generates <|X(ω)|2> . The program also generates the quantity <X(ω)> upon which we shall comment in Section 36 below. At this point, before Xpulse(ω) is added to the result, we plot <|X(ω)|2>. As expected, we see the peaks of δ6 spaced by 2π and having height around 10 units, Fig 35.1 We now insert copies of Xpulse(ω) as appropriate, and then we can plot for ω in the same range (-10,10) Fig 35.2 We see that the zeros of the sinc function have killed off the adjacent peaks. Next, we restrict the plot height to be 0.8 units to view the detail, chopping off the δ6 peak, Fig 35.3 The spectrum is seen to have a continuous component which very well approximates one quarter of the sinc2 curve, as we hoped it would. This tracking also occurs away from the central peak. Here is a blow-up of the above plot for ω in the range (5, 30) Fig 35.4 It might be noted that Maple does this work analytically, so that Was-av is a function of ω having a large number of trigonometric terms. For the reader's interest, we show Was-av(ω) for a typical program run : In more serious work with larger numbers, one would of course do this in a more numeric fashion, but we are able to confirm the basic results even with this small experiment. (f) What role has the Autocorrelation Function played in our development? In Section 32 the autocorrelation function was first introduced as rx(t) for x(t), and was computed for the simple case of a box pulse. Treating the definition of rx(t) as a convolution equation, the Wiener-Khintchine Relation Rx(ω) = |X(ω)|2 was trivially derived. It was then shown that the spectral energy density of a pulse train is E(ω) = (1/2π) Rx(ω) due to this relation. Finally, it was shown that rx(0) = E, the total energy in the pulse train. We then commented on the origin of the name, showing that the auto-correlation function is the cross-correlation function of a function with itself when that function is real. In Section 34 it was noted again that P = rx(0)/T since P = E/T, and that P(ω) = Rx(ω)/ (2πT) since Rx(ω) = |X(ω)|2. Sections 33 and 35 made no reference at all to the autocorrelation function. This leads us to make several comments: (1) The autocorrelation function played no role whatsoever in our development of key equations such as <P(ω)> = Ppulse(ω) [ (β-α) + α !Syntax Error, I2π δ(ωT1- 2πm) ] // infinite (35.11) (2) In some textbooks, one gets the impression that the autocorrelation function is somehow crucial for the development of such equations. It is not. (3) Nevertheless, since P(ω) = Rx(ω)/ (2πT), one can start with a description of x(i)(t) of pulse train i in a statistical ensemble, compute from it rx(i)(t) and from that Rx(i)(ω) . One could then do a statistical average to obtain <P(ω)> = (2πT)-1(1/I) Σi=1I Rx(i)(ω). So it is possible to take a pathway to deriving equations like (35.11) which does pass through the land of the autocorrelation function. (4) Our main reason for even bringing it up is that autocorrelation is closely related to what we are doing, and in other applications such as those involving noise (and pseudo-noise PN) it becomes more significant. In such applications, the definition of rx(t) might include a normalizing factor so it is then autocorrelation per unit time, or per pulse (per chip in the PN world). (5) In (34.14a) we noted that P(ω) = Ppulse(ω) R"(z) where R"(z) is the Z transform of the autocorrelation sequence rs defined in (32.17). Then in Appendix F (e) we compute R"(z) for a sequence which respects certain simple conditions met by a Maximum Length Sequence (MLS). The resulting P(ω) is not easily computed in any other manner, so in this case the pathway through the autocorrelation route is extremely valuable. (g) A paradox and its resolution Now while here, we can back up and apply our statistical average directly to the spectrum in (34.6) using (34.7). The result is <X(ω)> = Xpulse(ω) !Syntax Error, I<yn> e-inωT . (34.6,7) We could then argue that <yn> = [p] A + [1-p] B = p if A=1, B=0 . (35.23) In this case, we can extract p from the above sum, which then collapses to form the usual (13.2) delta function sum. The result is then the same as the regular pulse train result (14.4) with an overall factor of p out front, namely, <X(ω)> = p !Syntax Error, IXpulse(mω1) 2π δ( ωT1 - 2πm) . (35.24) This says that our average spectrum is 100% discrete, there is no continuous part! If p = 1/2, the average spectrum is just 1/2 times our discrete unit amplitude pulse train spectrum (14.4). How can this be true, if we just showed in (35.11) that the average spectral density <|X(ω)|2> has a continuous spectral component? The answer lies in the fact that <ab> ≠ <a><b>, where < > is our averaging operation. Thus <|X(ω)|2> ≠ <X(ω)><X(ω)*> . (35.25) There is no reason in the world why the average of a product should be the product of the averages, ( Σi=1N AiBi) ≠ ( Σi=1N Ai) ( Σi=1N Bi) . In the numerical example presented in Section 35 (f), we computed <X(ω)> for a small ensemble of pulse trains. Here are plots of the real and imaginary part of <X(ω)>, Fig 35.5 We can see that, apart from the noise of our low statistics, there is only the central peak and no continuous spectrum component. A blow-up of the central region follows, Fig 35.6