Home / Math and Physics Files / Math / Probability and Statistics

Ch 2- Digital Communications J. Proakis 5th_Edition 2007-3

PDF · 78 pages · 541.1 KB

Textbook chapter by John Proakis, kept in the archive's probability and statistics folder. It opens with tables of Fourier transform properties and pairs, then covers bandpass and lowpass signal representation, analytic signals, the Hilbert transform, lowpass equivalents, and in-phase and quadrature components. The chapter also reviews random variables, random processes, bandpass random processes, and series expansions of random processes. Only the opening part was read.

AI-written summary; may contain errors.

Extracted text (machine-read; may contain errors)

Proakis-27466 book September 25, 2007 13:9 2 Deterministic and Random Signal Analysis In this chapter we present the background material needed in the study of the following chapters. The analysis of deterministic and random signals and the study of different methods for their representation are the main topics of this chapter. In addition, wealso introduce and study the main properties of some random variables frequentlyencountered in analysis of communication systems. We continue with a review ofrandom processes, properties of lowpass and bandpass random processes, and seriesexpansion of random processes. Throughout this chapter, and the book, we assume that the reader is familiar with the properties of the Fourier transform as summarized in Table 2.0–1 and the importantFourier transform pairs given in Table 2.0–2. In these tables we have used the following signal deﬁnitions. /Pi1(t)= ⎧ ⎪⎨ ⎪⎩1|t|<1 2 1 2t=±1 2 0 otherwisesinc(t )=⎦braceleftBiggsin(π t) πtt/negationslash=0 1 t=0 and sgn(t )=⎧ ⎪⎨ ⎪⎩1 t>0 −1 t<0 0 t=0/Lambda1(t)=/Pi1(t)⋆/Pi1(t)=⎧ ⎪⎨ ⎪⎩t+1−1≤t<0 −t+10≤t<1 0 otherwise The unit step signal u−1(t) is deﬁned as u−1(t)=⎧ ⎪⎨ ⎪⎩1t>0 1 2t=0 0t<0 We also assume that the reader is familiar with elements of probability, random variables, and random processes as covered in standard texts such as Papoulis and Pillai (2002), Leon-Garcia (1994), and Stark and Woods (2002). 17 Proakis-27466 book September 25, 2007 13:9 18 Digital Communications TABLE 2.0–1 Table of Fourier Transform Properties Property Signal Fourier Transform Linearity αx1(t)+βx2(t) αX1(f)+βX2(f) Duality X(t) x(−f) Conjugacy x∗(t) X∗(−f) Time-scaling ( a/negationslash=0) x(at)1 |a|X⎦parenleftbigf a⎦parenrightbig Time-shift x(t−t0) e−j2πft0X(f) Modulation ej2πf0tx(t) X(f−f0) Convolution x(t)⋆y(t) X(f)Y(f) Multiplication x(t)y(t) X(f)⋆Y(f) Differentiationdn dtnx(t)( j2πf)nX(f) Differentiation in frequency tnx(t)⎦parenleftbigj 2π⎦parenrightbigndn dfnX(f) Integration⎦integraldisplayt −∞x(τ)dτX(f) j2πf+1 2X(0)δ(f) Parseval’s theorem⎦integraldisplay∞ −∞x(t)y∗(t)dt=⎦integraldisplay∞ −∞X(f)Y∗(f)df Rayleigh’s theorem⎦integraldisplay∞ −∞|x(t)|2dt=⎦integraldisplay∞ −∞|X(f)|2df 2.1 BANDPASS AND LOWPASS SIGNAL REPRESENTATION As was discussed in Chap. 1, the process of communication consists of transmission of the output of an information source over a communication channel. In almost allcases, the spectral characteristics of the information sequence do not directly match thespectral characteristics of the communication channel, and hence the information signalcannot be directly transmitted over the channel. In many cases the information signalis a low frequency (baseband) signal, and the available spectrum of the communicationchannel is at higher frequencies. Therefore, at the transmitter the information signal istranslated to a higher frequency signal that matches the properties of the communicationchannel. This is the modulation process in which the baseband information signal isturned into a bandpass modulated signal. In this section we study the main propertiesof baseband and bandpass signals. 2.1–1 Bandpass and Lowpass Signals In this section we will show that any real, narrowband, and high frequency signal—called a bandpass signal—can be represented in terms of a complex low frequency Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 19 TABLE 2.0–2 Table of Fourier Transform Pairs Time Domain Frequency Domain δ(t)1 1 δ(f) δ(t−t0) e−j2πft0 ej2πf0tδ(f−f0) cos(2πf0t)1 2δ(f−f0)+1 2δ(f+f0) sin(2πf0t)1 2jδ(f−f0)−1 2jδ(f+f0) /Pi1(t) sinc( f) sinc( t) /Pi1(f) /Lambda1(t) sinc2(f) sinc2(t) /Lambda1(f) e−αtu−1(t),α> 01 α+j2πf te−αtu−1(t),α> 01 (α+j2πf)2 e−α|t|(α> 0)2α α2+(2πf)2 e−πt2e−πf2 sgn(t)1 jπf u−1(t)1 2δ(f)+1 j2πf 1 2δ(t)+j1 2πtu−1(f) δ/prime(t) j2πf δ(n)(t)( j2πf)n 1 t−jπsgn( f) ∞⎦summationdisplay n=−∞δ(t−nT0)1 T0∞⎦summationdisplay n=−∞δ⎦parenleftBig f−n T0⎦parenrightBig signal, called the lowpass equivalent of the original bandpass signal. This result makes it possible to work with the lowpass equivalents of bandpass signals instead of directlyworking with them, thus greatly simplifying the handling of bandpass signals. That isso because applying signal processing algorithms to lowpass signals is much easier dueto lower required sampling rates which in turn result in lower rates of the sampled data. The Fourier transform of a signal provides information about the frequency content, orspectrum , of the signal. The Fourier transform of a real signal x(t) has Hermitian symmetry , i.e., X(−f)=X ∗(f), from which we conclude that |X(−f)|=| X(f)|and /negationslashX∗(f)=− /negationslashX(f). In other words, for real x(t), the magnitude of X(f)i se v e na n d Proakis-27466 book September 25, 2007 13:9 20 Digital Communications WX(f) fFIGURE 2.1–1 The spectrum of a real-valued lowpass(baseband) signal. its phase is odd. Because of this symmetry, all information about the signal is in the positive (or negative) frequencies, and in particular x(t) can be perfectly reconstructed by specifying X(f) for f≥0. Based on this observation, for a real signal x(t), we deﬁne the bandwidth as the smallest range of positive frequencies such that X(f)=0 when|f|is outside this range. It is clear that the bandwidth of a real signal is one-half of its frequency support set. Alowpass ,o rbaseband , signal is a signal whose spectrum is located around the zero frequency. For instance, speech, music, and video signals are all lowpass signals,although they have different spectral characteristics and bandwidths. Usually lowpasssignals are low frequency signals, which means that in the time domain, they are slowlyvarying signals with no jumps or sudden variations. The bandwidth of a real lowpasssignal is the minimum positive Wsuch that X(f)=0 outside [ −W,+W]. For these signals the frequency support , i.e., the range of frequencies for which X(f)/negationslash=0, is [−W,+W]. An example of the spectrum of a real-valued lowpass signal is shown in Fig. 2.1–1. The solid line shows the magnitude spectrum |X(f)|, and the dashed line indicates the phase spectrum /negationslashX(f). We also deﬁne the positive spectrum and the negative spectrum of a signal x(t)a s X+(f)=⎧ ⎪⎪⎨ ⎪⎪⎩X(f) f>0 1 2X(0) f=0 0 f<0X−(f)=⎧ ⎪⎪⎨ ⎪⎪⎩X(f) f<0 1 2X(0) f=0 0 f>0(2.1–1) It is clear that X+(f)=X(f)u−1(f),X−(f)=X(f)u−1(−f) and X(f)=X+(f)+ X−(f). For a real signal x(t), since X(f) is Hermitian, we have X−(f)=X∗ +(−f). For a complex signal x(t), the spectrum X(f) is not symmetric; hence, the signal cannot be reconstructed from the information in the positive frequencies only. Forcomplex signals, we deﬁne the bandwidth as one-half of the entire range of frequencies over which the spectrum is nonzero, i.e., one-half of the frequency support of the signal. This deﬁnition is for consistency with the deﬁnition of bandwidth for real signals. Withthis deﬁnition we can state that in general and for all signals, real or complex, thebandwidth is deﬁned as one-half of the frequency support. In practice, the spectral characteristics of the message signal and the communication channel do not always match, and it is required that the message signal be modulated by one of the many different modulation methods to match its spectral characteristics to Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 21 X(f) X/H11002(f) X/H11001(f) f f0 /H11002f0 FIGURE 2.1–2 The spectrum of a real-valued bandpass signal. the spectral characteristics of the channel. In this process, the spectrum of the lowpass message signal is translated to higher frequencies. The resulting modulated signal is abandpass signal. Abandpass signal is a real signal whose frequency content, or spectrum, is located around some frequency ±f 0which is far from zero. More formally, we deﬁne a bandpass signal to be a real signal x(t) for which there exists positive f0andWsuch that the positive spectrum of X(f), i.e., X+(f), is nonzero only in the interval [ f0−W/2,f0+ W/2], where W/2<f0(in practice, usually W/lessmuchf0). The frequency f0is called the central frequency. Obviously, the bandwidth of x(t) is at most equal to W. Bandpass signals are usually high frequency signals which are characterized by rapid variationsin the time domain. An example of the spectrum of a bandpass signal is shown in Figure 2.1–2. Note that since the signal x(t) is real, its magnitude spectrum (solid line) is even, and its phase spectrum (dashed line) is odd. Also, note that the central frequency f 0is not necessarily the midband frequency of the bandpass signal. Due to the symmetry of the spectrum, X+(f) has all the information that is necessary to reconstruct X(f). In fact we can write X(f)=X+(f)+X−(f)=X+(f)+X∗ +(−f) (2.1–2) which means that knowledge of X+(f) is sufﬁcient to reconstruct X(f). 2.1–2 Lowpass Equivalent of Bandpass Signals We start by deﬁning the analytic signal ,o rt h epre-envelope, corresponding to x(t)a s the signal x+(t) whose Fourier transform is X+(f). This signal contains only positive frequency components, and its spectrum is not Hermitian. Therefore, in general, x+(t) is a complex signal. We have x+(t)=F−1[X+(f)] =F−1[X(f)u−1(f)] =x(t)⋆⎦parenleftbigg1 2δ(t)+j1 2πt⎦parenrightbigg =1 2x(t)+j 2⎦hatwidex(t)(2.1–3) Proakis-27466 book September 25, 2007 13:9 22 Digital Communications Xl(f) /H11005 2X/H11001(f /H11001 f0 ) fFIGURE 2.1–3 The spectrum of the lowpass equivalent of thesignal shown in Figure 2.1–2. where ⎦hatwidex(t)=1 πt⋆x(t)i st h e Hilbert transform ofx(t). The Hilbert transform of x(t)i s obtained by introducing a phase shift of −π 2at positive frequency components of x(t) andπ 2at negative frequencies. In the frequency domain we have F⎦bracketleftbig⎦hatwidex(t)⎦bracketrightbig=− jsgn( f)X(f) (2.1–4) Some of the properties of the Hilbert transform will be covered in the problems at the end of this chapter. Now we deﬁne xl(t), the lowpass equivalent , or the complex envelope ,o fx(t), as the signal whose spectrum is given by 2 X+(f+f0), i.e., Xl(f)=2X+(f+f0)=2X(f+f0)u−1(f+f0) (2.1–5) Obviously the spectrum of xl(t) is located around the zero frequency, and therefore it is in general a complex lowpass signal. This signal is called the lowpass equivalent or the complex envelope ofx(t). The spectrum of the lowpass equivalent of the signal shown in Figure 2.1–2 is shown in Figure 2.1–3. Applying the modulation theorem of the Fourier transform, we obtain xl(t)=F−1[Xl(f)] =2x+(t)e−j2πf0t =(x(t)+j⎦hatwidex(t))e−j2πf0t(2.1–6) =(x(t) cos 2 πf0t+⎦hatwidex(t) sin 2πf0t) +j(⎦hatwidex(t) cos 2 πf0t−x(t) sin 2πf0t) (2.1–7) From Equation 2.1–6 we can write x(t)=Re⎦bracketleftbigxl(t)ej2πf0t⎦bracketrightbig(2.1–8) This relation expresses any bandpass signals in terms of its lowpass equivalent. Using Equations 2.1–2 and 2.1–5, we can write X(f)=1 2⎦bracketleftbigXl(f−f0)+X∗ l(−f−f0)⎦bracketrightbig(2.1–9) Equations 2.1–8, 2.1–9, 2.1–5, and 2.1–7 express x(t) and xl(t) in terms of each other in the time and frequency domains. The real and imaginary parts of xl(t) are called the in-phase component and the quadrature component ofx(t), respectively, and are denoted by xi(t) and xq(t). Both xi(t) and xq(t) are real-valued lowpass signals, and we have xl(t)=xi(t)+jxq(t) (2.1–10) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 23 Comparing Equations 2.1–10 and 2.1–7, we conclude that xi(t)=x(t) cos 2 πf0t+⎦hatwidex(t) sin 2πf0t xq(t)=⎦hatwidex(t) cos 2 πf0t−x(t) sin 2πf0t(2.1–11) Solving Equation 2.1–11 for x(t) and⎦hatwidex(t)g i v e s x(t)=xi(t) cos 2 πf0t−xq(t) sin 2πf0t ⎦hatwidex(t)=xq(t) cos 2 πf0t+xi(t) sin 2πf0t(2.1–12) Equation 2.1–12 shows that any bandpass signal x(t) can be expressed in terms of two lowpass signals, namely, its in-phase and quadrature components. Equation 2.1–10 expresses xl(t) in terms of its real and complex parts. We can write a similar relation in polar coordinates expressing x(t) in terms of its magnitude and phase. If we deﬁne the envelope andphase ofx(t), denoted by rx(t) andθx(t), respectively, by rx(t)=⎦radicalBig x2 i(t)+x2q(t) (2.1–13) θx(t)=arctanxq(t) xi(t)(2.1–14) we have xl(t)=rx(t)ejθx(t)(2.1–15) Substituting this result into Equation 2.1–8 gives x(t)=Re⎦bracketleftbigrx(t)ej(2πf0t+θx(t))⎦bracketrightbig(2.1–16) resulting in x(t)=rx(t) cos (2πf0t+θx(t)) (2.1–17) A bandpass signal and its envelope are shown in Figure 2.1–4. FIGURE 2.1–4 A bandpass signal. The dashed curve denotes the envelope. Proakis-27466 book September 25, 2007 13:9 24 Digital Communications It is important to note that xl(t)—and consequently xi(t),xq(t),rx(t), and θx(t)— depends on the choice of the central frequency f0. For a given bandpass signal x(t), different values of f0—as long as X+(f) is nonzero only in the interval [ f0−W/2,f0+ W/2], where W/2<f0—yield different lowpass signals xl(t). Therefore, it makes more sense to deﬁne the lowpass equivalent of a bandpass signal with respect to aspeciﬁc f 0. Since in most cases the choice of f0is clear, we usually do not make this distinction. Equations 2.1–12 and 2.1–17 provide two methods for representing a bandpass signal x(t) in terms of two lowpass signals, one in terms of the in-phase and quadrature components and one in terms of the envelope and the phase. The two relations given inEquations 2.1–8 and 2.1–12 that express the bandpass signal in terms of the lowpasscomponent(s) deﬁne the modulation process, i.e., the process of going from lowpass tobandpass. The system that implements this process is called a modulator . The structure of a general modulator implementing Equations 2.1–8 and 2.1–12 is shown in Fig-ure 2.1–5(a) and (b). In this ﬁgure double lines and double blocks indicate complexvalues and operations. Similarly, Equations 2.1–7 and 2.1–11 represent how x l(t), or xi(t) and xq(t), can be obtained from the bandpass signal x(t). This process, i.e., extracting the lowpass signal from the bandpass signal, is called the demodulation process and is shown in Figure 2.1–6(a) and (b). In these block diagrams the block denoted by Hrepresents a Hilbert transform, i.e., an LTI system with impulse response h(t)=1 πtand transfer function H(f)=− jsgn( f). xl(t) x(t)/H11003 /H11003/H11001xl(t) x(t)xi(t) xq(t)cos 2/H9266f0t /H11002sin 2/H9266f0t/H11003xl(t) 2x/H11001(t)x(t) Re(·) f0 Modulator(a) (b) (c)ej2/H9266f0t FIGURE 2.1–5 A complex (a) and real (b) modulator. A general representation for a modulator isshown in (c). Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 25 x(t) xl(t)f0 Demodulator(a) (b) (c)xi(t) xq(t)x(t)cos 2/H9266 f0t /H11002sin 2/H9266 f0t/H11003 /H11001 /H11001sin 2/H9266 f0t cos 2/H9266 f0t/H11003 /H11003 /H11003/H5108/H11003xl(t) x(t)e/H11002j2/H9266f0t 2x/H11001(t)/H11001 /H11003 /H5108 xˆ(t) jxˆ(t) FIGURE 2.1–6 A complex (a) and real (b) demodulator. A general representation for a demodulator isshown in (c). 2.1–3 Energy Considerations In this section we study the relation between energy contents of the signals introduced in the preceding pages. The energy of a signal x(t) is deﬁned as Ex=⎦integraldisplay∞ −∞|x(t)|2dt (2.1–18) and by Rayleigh’s relation from Table 2.0–1 we can write Ex=⎦integraldisplay∞ −∞|x(t)|2dt=⎦integraldisplay∞ −∞|X(f)|2dt (2.1–19) Since there is no overlap between X+(f) and X−(f), we have X+(f)X−(f)=0, and hence Ex=⎦integraldisplay∞ −∞|X+(f)+X−(f)|2df =⎦integraldisplay∞ −∞|X+(f)|2df+⎦integraldisplay∞ −∞|X−(f)|2df =2⎦integraldisplay∞ −∞|X+(f)|2df =2Ex+(2.1–20) Proakis-27466 book September 25, 2007 13:9 26 Digital Communications On the other hand, Ex=2⎦integraldisplay∞ −∞|X+(f)|2df =2⎦integraldisplay∞ −∞⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingleXl(f) 2⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle2 df =1 2Exl(2.1–21) This shows that the energy in the lowpass equivalent signal is twice the energy in the bandpass signal. We deﬁne the inner product of two signals x (t) and y(t)a s /angbracketleftx(t),y(t)/angbracketright=⎦integraldisplay∞ −∞x(t)y∗(t)dt=⎦integraldisplay∞ −∞X(f)Y∗(f)df (2.1–22) where we have used Parseval’s relation from Table 2.0–1. Obviously Ex=/angbracketleftx(t),x(t)/angbracketright (2.1–23) In Problem 2.2 we prove that if x(t) and y(t) are two bandpass signals with lowpass equivalents xl(t) and yl(t) with respect to the same f0, then /angbracketleftx(t),y(t)/angbracketright=1 2Re[/angbracketleftxl(t),yl(t)/angbracketright] (2.1–24) The complex quantity ρx,y, called the cross-correlation coefﬁcient ofx(t) and y(t), is deﬁned as ρx,y=/angbracketleftx(t),y(t)/angbracketright⎦radicalbigExEy(2.1–25) and represents the normalized inner product between two signals. From Exl=2Exand Equation 2.1–24 we can conclude that if x(t) and y(t) are bandpass signals with the same f0, then ρx,y=Re (ρxl,yl) (2.1–26) Two signals are orthogonal if their inner product (and subsequently, their ρ)i s zero. Note that if ρxl,yl=0, then using Equation 2.1–26, we have ρx,y=0; but the converse is not necessarily true. In other words, orthogonality in the baseband implies orthogonality in the pass band, but not vice versa . EXAMPLE 2.1–1. Assume that m(t) is a real baseband signal with bandwidth W,and deﬁne two signals x(t)=m(t) cos 2 πf0tandy(t)=m(t) sin 2πf0t, where f0>W. Comparing these relations with Equation 2.1–12, we conclude that xi(t)=m(t) xq(t)=0 yi(t)=0 yq(t)=−m(t) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 27 or, equivalently, xl(t)=m(t) yl(t)=− jm(t) Note that here ρxl,yl=j⎦integraldisplay∞ −∞m2(t)=jEm Therefore, ρx,y=Re (ρ xl,yl)=Re (jEm)=0 This means that x(t) and y(t) are orthogonal, but their lowpass equivalents are not orthogonal. 2.1–4 Lowpass Equivalent of a Bandpass System A bandpass system is a system whose transfer function is located around a frequency f0(and its mirror image −f0). More formally, we deﬁne a bandpass system as a system whose impulse response h(t) is a bandpass signal. Since h(t) is bandpass, it has a lowpass equivalent denoted by hl(t) where h(t)=Re⎦bracketleftbighl(t)ej2πf0t⎦bracketrightbig(2.1–27) If a bandpass signal x(t) passes through a bandpass system with impulse response h(t), then obviously the output will be a bandpass signal y(t). The relation between the spectra of the input and the output is given by Y(f)=X(f)H(f) (2.1–28) Using Equation 2.1–5, we have Yl(f)=2Y(f+f0)u−1(f+f0) =2X(f+f0)H(f+f0)u−1(f+f0) =1 2[2X(f+f0)u−1(f+f0)][2H(f+f0)u−1(f+f0)] =1 2Xl(f)Hl(f) (2.1–29) where we have used the fact that for f>−f0, which is the range of frequencies of interest, u2 −1(f+f0)=u−1(f+f0)=1. In the time domain we have yl(t)=1 2xl(t)⋆hl(t) (2.1–30) Equations 2.1–29 and 2.1–30 show that when a bandpass signal passes through a bandpass system, the input-output relation between the lowpass equivalents is verysimilar to the relation between the bandpass signals, the only difference being that forthe lowpass equivalents a factor of 1 2is introduced. Proakis-27466 book September 25, 2007 13:9 28 Digital Communications 2.2 SIGNAL SPACE REPRESENTATION OF WA VEFORMS Signal space (or vector) representation of signals is a very effective and useful tool in the analysis of digitally modulated signals. We cover this important approach in thissection and show that any set of signals is equivalent to a set of vectors. We show thatsignals have the same basic properties of vectors. We study methods of determining anequivalent set of vectors for a set of signals and introduce the notion of signal spacerepresentation, or signal constellation, of a set of waveforms. 2.2–1 Vector Space Concepts A vector vin an n-dimensional space is characterized by its ncomponents v1v2···v n. Letvdenote a column vector, i.e., v=[v1v2···v n]t, where Atdenotes the transpose of matrix A. The inner product of two n-dimensional vectors v1=[v11v12···v 1n]t andv2=[v21v22···v 2n]tis deﬁned as /angbracketleftv1,v2/angbracketright=v 1·v2=n⎦summationdisplay i=1v1iv∗ 2i=vH 2v1 (2.2–1) where AHdenotes the Hermitian transpose of the matrix A, i.e., the result of ﬁrst transposing the matrix and then conjugating its elements. From the deﬁnition of theinner product of two vectors it follows that /angbracketleftv 1,v2/angbracketright=/angbracketleftv2,v1/angbracketright∗(2.2–2) and therefore, /angbracketleftv1,v2/angbracketright+/angbracketleftv2,v1/angbracketright=2R e [/angbracketleftv1,v2/angbracketright] (2.2–3) A vector may also be represented as a linear combination of orthogonal unit vectors or an orthonormal basis ei,1≤i≤n, i.e., v=n⎦summationdisplay i=1viei (2.2–4) where, by deﬁnition, a unit vector has length unity and viis the projection of the vector vonto the unit vector ei, i.e.,vi=/angbracketleftv,ei/angbracketright. Two vectors v1andv2areorthogonal if /angbracketleftv1,v2/angbracketright=0. More generally, a set of mvectors vk,1≤k≤m, are orthogonal if /angbracketleftvi,vj/angbracketright=0 for all 1 ≤i,j≤m, and i/negationslash=j. The norm of a vector vis denoted by /bardblv/bardbl and is deﬁned as /bardblv/bardbl=(/angbracketleftv ,v/angbracketright)1/2=⎦radicaltp⎦radicalvertex⎦radicalvertex⎦radicalbtn⎦summationdisplay i=1|vi|2 (2.2–5) which in the n-dimensional space is simply the length of the vector. A set of mvec- tors is said to be orthonormal if the vectors are orthogonal and each vector has a Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 29 unit norm. A set of mvectors is said to be linearly independent if no one vector can be represented as a linear combination of the remaining vectors. Any two n-dimensional vectors v1andv2satisfy the triangle inequality /bardblv1+v2/bardbl≤/bardblv1/bardbl+/bardblv2/bardbl (2.2–6) with equality if v1andv2are in the same direction, i.e., v1=av2where ais a positive real scalar. The Cauchy–Schwarz inequality states that |/angbracketleftv1,v2/angbracketright |≤/bardblv1/bardbl·/bardblv2/bardbl (2.2–7) with equality if v1=av2for some complex scalar a. The norm square of the sum of two vectors may be expressed as /bardblv1+v2/bardbl2=/bardblv1/bardbl2+/bardblv2/bardbl2+2R e [/angbracketleftv1,v2/angbracketright] (2.2–8) Ifv1andv2are orthogonal, then /angbracketleftv1,v2/angbracketright=0 and, hence, /bardblv1+v2/bardbl2=/bardblv1/bardbl2+/bardblv2/bardbl2(2.2–9) This is the Pythagorean relation for two orthogonal n-dimensional vectors. From matrix algebra, we recall that a linear transformation in an n-dimensional vector space is a matrix transformation of the form v/prime=Av, where the matrix Atransforms the vector vinto some vector v/prime. In the special case where v/prime=λv, i.e., Av=λv where λis some scalar, the vector vis called an eigenvector of the transformation and λis the corresponding eigenvalue . Finally, let us review the Gram–Schmidt procedure for constructing a set of or- thonormal vectors from a set of n-dimensional vectors vi,1≤i≤m. We begin by arbitrarily selecting a vector from the set, say, v1. By normalizing its length, we obtain the ﬁrst vector, say, u1=v1 /bardblv1/bardbl(2.2–10) Next, we may select v2and, ﬁrst, subtract the projection of v2onto u1. Thus, we obtain u/prime 2=v2−(/angbracketleftv2,u1/angbracketright)u1 (2.2–11) Then we normalize the vector u/prime 2to unit length. This yields u2=u/prime 2 /bardblu/prime 2/bardbl(2.2–12) The procedure continues by selecting v3and subtracting the projections of v3intou1 andu2. Thus, we have u/prime 3=v3−(/angbracketleftv3,u1/angbracketright)u1−(/angbracketleftv3,u2/angbracketright)u2 (2.2–13) Then the orthonormal vector u3is u3=u/prime 3 /bardblu/prime 3/bardbl(2.2–14) Proakis-27466 book September 25, 2007 13:9 30 Digital Communications By continuing this procedure, we construct a set of Northonormal vectors, where N≤min(m ,n). 2.2–2 Signal Space Concepts As in the case of vectors, we may develop a parallel treatment for a set of signals. The inner product of two generally complex-valued signals x1(t) and x2(t) is denoted by /angbracketleftx1(t),x2(t)/angbracketrightand deﬁned as /angbracketleftx1(t),x2(t)/angbracketright=⎦integraldisplay∞ −∞x1(t)x∗ 2(t)dt (2.2–15) similar to Equation 2.1–22. The signals are orthogonal if their inner product is zero. Thenorm of a signal is deﬁned as /bardblx(t)/bardbl=⎦parenleftbigg⎦integraldisplay∞ −∞|x(t)|2dt⎦parenrightbigg1/2 =⎦radicalbig Ex (2.2–16) where Exis the energy in x(t). A set of msignals is orthonormal if they are orthogonal and their norms are all unity. A set of msignals is linearly independent if no signal can be represented as a linear combination of the remaining signals. The triangle inequality for two signals is simply /bardblx1(t)+x2(t)/bardbl≤/bardbl x1(t)/bardbl+/bardbl x2(t)/bardbl (2.2–17) and the Cauchy–Schwarz inequality is |/angbracketleftx1(t),x2(t)/angbracketright|≤/bardblx1(t)/bardbl·/bardblx2(t)/bardbl=⎦radicalBig Ex1Ex2 (2.2–18) or, equivalently, ⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦integraldisplay ∞ −∞x1(t)x∗ 2(t)dt⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle ≤⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦integraldisplay ∞ −∞|x1(t)|2dt⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle1/2⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦integraldisplay ∞ −∞|x2(t)|2dt⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle1/2 (2.2–19) with equality when x2(t)=ax1(t), where ais any complex number. 2.2–3 Orthogonal Expansions of Signals In this section, we develop a vector representation for signal waveforms, and thus we demonstrate an equivalence between a signal waveform and its vector representation.Suppose that s(t) is a deterministic signal with ﬁnite energy Es=⎦integraldisplay∞ −∞|s(t)|2dt (2.2–20) Furthermore, suppose that there exists a set of functions {φn(t),n=1,2,..., K}that are orthonormal in the sense that /angbracketleftφn(t),φ m(t)/angbracketright=⎦integraldisplay∞ −∞φn(t)φ∗ m(t)dt=⎦braceleftBigg 1m=n 0m/negationslash=n(2.2–21) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 31 We may approximate the signal s(t) by a weighted linear combination of these func- tions, i.e., ⎦hatwides(t)=K⎦summationdisplay k=1skφk(t) (2.2–22) where {sk,1≤k≤K}are the coefﬁcients in the approximation of s(t). The approx- imation error incurred is e(t)=s(t)−⎦hatwides(t) Let us select the coefﬁcients {sk}so as to minimize the energy Eeof the approximation error. Thus, Ee=⎦integraldisplay∞ −∞|s(t)−⎦hatwides(t)|2dt (2.2–23) =⎦integraldisplay∞ −∞⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingles(t)−K⎦summationdisplay k=1skφk(t)⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle2 dt (2.2–24) The optimum coefﬁcients in the series expansion of s(t) may be found by differentiating Equation 2.2–23 with respect to each of the coefﬁcients {sk}and setting the ﬁrst deriva- tives to zero. Alternatively, we may use a well-known result from estimation theorybased on the mean square error criterion, which, simply stated, is that the minimumof Eewith respect to the {sk}is obtained when the error is orthogonal to each of the functions in the series expansion. Thus, ⎦integraldisplay∞ −∞⎦bracketleftBigg s(t)−K⎦summationdisplay k=1skφk(t)⎦bracketrightBigg φ∗ n(t)dt=0, n=1,2,..., K (2.2–25) Since the functions {φn(t)}are orthonormal, Equation 2.2–25 reduces to sn=/angbracketlefts(t),φn(t)/angbracketright=⎦integraldisplay∞ −∞s(t)φ∗ n(t)dt, n=1,2,..., K (2.2–26) Thus, the coefﬁcients are obtained by projecting the signal s(t) onto each of the functions {φn(t)}. Consequently, ⎦hatwides(t) is the projection of s(t) onto the K-dimensional signal space spanned by the functions {φn(t)}, and therefore it is orthogonal to the error signal e(t)=s(t)−⎦hatwides(t), i.e., /angbracketlefte(t),⎦hatwides(t)/angbracketright=0. The minimum mean-square approxima- tion error is Emin=⎦integraldisplay∞ −∞e(t)s∗(t)dt (2.2–27) =⎦integraldisplay∞ −∞|s(t)|2dt−⎦integraldisplay∞ −∞K⎦summationdisplay k=1skφk(t)s∗(t)dt (2.2–28) =Es−K⎦summationdisplay k=1|sk|2(2.2–29) Proakis-27466 book September 25, 2007 13:9 32 Digital Communications which is nonnegative, by deﬁnition. When the minimum mean square approximation errorEmin=0, Es=K⎦summationdisplay k=1|sk|2=⎦integraldisplay∞ −∞|s(t)|2dt (2.2–30) Under the condition that Emin=0, we may express s(t)a s s(t)=K⎦summationdisplay k=1skφk(t) (2.2–31) where it is understood that equality of s(t) to its series expansion holds in the sense that the approximation error has zero energy. When every ﬁnite energy signal can be represented by a series expansion of the form in Equation 2.2–31 for which Emin=0, the set of orthonormal functions {φn(t)} is said to be complete . EXAMPLE 2.2–1. TRIGONOMETRIC FOURIER SERIES: Consider a ﬁnite energy real sig- nals(t) that is zero everywhere except in the range 0 ≤t≤Tand has a ﬁnite number of discontinuities in this interval. Its periodic extension can be represented in a Fourierseries as s(t)=∞⎦summationdisplay k=0⎦parenleftbigg akcos2πkt T+bksin2πkt T⎦parenrightbigg (2.2–32) where the coefﬁcients {ak,bk}that minimize the mean square error are given by a0=1 T⎦integraldisplayT 0s(t)dt ak=2 T⎦integraldisplayT 0s(t) cos2πkt Tdt,k=1,2,3,... bk=2 T⎦integraldisplayT 0s(t) sin2πkt Tdt,k=1,2,3,...(2.2–33) The set of functions {1/√ T,√2/Tcos 2πkt/T,√2/Tsin 2πkt/T}is a complete set for the expansion of periodic signals on the interval [0 ,T], and, hence, the series expansion results in zero mean square error. EXAMPLE 2.2–2. EXPONENTIAL FOURIER SERIES: Consider a general ﬁnite energy sig- nals(t) (real or complex) that is zero everywhere except in the range 0 ≤t≤Tand has a ﬁnite number of discontinuities in this interval. Its periodic extension can berepresented in an exponential Fourier series as s(t)=∞⎦summationdisplay n=−∞xnej2πn Tt(2.2–34) where the coefﬁcients {xn}that minimize the mean square error are given by xn=1 T⎦integraldisplay∞ −∞x(t)e−j2πn Ttdt (2.2–35) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 33 The set of functions {√1/Tej2πn Tt}is a complete set for expansion of periodic signals on the interval [0, T], and, hence, the series expansion results in zero mean square error. 2.2–4 Gram-Schmidt Procedure Now suppose that we have a set of ﬁnite energy signal waveforms {sm(t),m=1,2,..., M}and we wish to construct a set of orthonormal waveforms. The Gram-Schmidt orthogonalization procedure allows us to construct such a set. This procedure is similar to the one described in Section 2.2–1 for vectors. We begin with the ﬁrst waveform s1(t), which is assumed to have energy E1. The ﬁrst orthonormal waveform is simply constructed as φ1(t)=s1(t)√E1(2.2–36) Thus, φ1(t) is simply s1(t) normalized to unit energy. The second waveform is con- structed from s2(t) by ﬁrst computing the projection of s2(t) onto φ1(t), which is c21=/angbracketlefts2(t),φ 1(t)/angbracketright=⎦integraldisplay∞ −∞s2(t)φ∗ 1(t)dt (2.2–37) Then c21φ1(t) is subtracted from s2(t) to yield γ2(t)=s2(t)−c21φ1(t) (2.2–38) This waveform is orthogonal to φ1(t), but it does not have unit energy. If E2denotes the energy of γ2(t), i.e., E2=⎦integraldisplay∞ −∞γ2 2(t)dt the normalized waveform that is orthogonal to φ1(t)i s φ2(t)=γ2(t)√E2(2.2–39) In general, the orthogonalization of the kth function leads to φk(t)=γk(t)√Ek(2.2–40) where γk(t)=sk(t)−k−1⎦summationdisplay i=1ckiφi(t) (2.2–41) cki=/angbracketleftsk(t),φ i(t)/angbracketright=⎦integraldisplay∞ −∞sk(t)φ∗ i(t)dt, i=1,2,...,k −1 (2.2–42) Ek=⎦integraldisplay∞ −∞γ2 k(t)dt (2.2–43) Proakis-27466 book September 25, 2007 13:9 34 Digital Communications Thus, the orthogonalization process is continued until all the Msignal waveforms {sm(t)}have been exhausted and N≤Morthonormal waveforms have been con- structed. The dimensionality Nof the signal space will be equal to Mif all the signal waveforms are linearly independent, i.e., none of the signal waveforms is a linearcombination of the other signal waveforms. EXAMPLE 2.2–3. Let us apply the Gram-Schmidt procedure to the set of four wave- forms illustrated in Figure 2.2–1. The waveform s1(t) has energy E1=2, so that φ1(t)=⎦radicalbigg 1 2s1(t) Next we observe that c21=0; hence, s2(t) andφ1(t) are orthogonal. Therefore, φ2(t)= s2(t)/√E2=⎦radicalBig 1 2s2(t). To obtain φ3(t), we compute c31andc32, which are c31=√ 2 andc23=0. Thus, γ3(t)=s3(t)−√ 2φ1(t)=⎦braceleftbigg−12 ≤t≤3 0 otherwise Since γ3(t) has unit energy, it follows that φ3(t)=γ3(t). Determining φ4(t), we ﬁnd thatc41=−√ 2,c42=0, and c43=1. Hence, γ4(t)=s4(t)+√ 2φ1(t)−φ3(t)=0 Consequently, s4(t) is a linear combination of φ1(t) andφ3(t) and, hence, φ4(t)=0. The three orthonormal functions are illustrated in Figure 2.2–1(b). Once we have constructed the set of orthonormal waveforms {φn(t)}, we can express theMsignals {sm(t)}as linear combinations of the {φn(t)}. Thus, we may write sm(t)=N⎦summationdisplay n=1smnφn(t), m=1,2,..., M (2.2–44) Based on the expression in Equation 2.2–44, each signal may be represented by the vector sm=[sm1sm2···smN]t(2.2–45) or, equivalently, as a point in the N-dimensional (in general, complex) signal space with coordinates {smn,n=1,2,..., N}. Therefore, a set of Msignals {sm(t)}M m=1can be represented by a set of Mvectors {sm}M m=1in the N-dimensional space, where N≤M. The corresponding set of vectors is called the signal space representation ,o rcon- stellation ,o f{sm(t)}M m=1. If the original signals are real, then the corresponding vector representations are in RN; and if the signals are complex, then the vector representations are in CN. Figure 2.2–2 demonstrates the process of obtaining the vector equivalent from a signal (signal-to-vector mapping) and vice versa (vector-to-signal mapping). From the orthonormality of the basis {φn(t)}it follows that Em=⎦integraldisplay∞ −∞|sm(t)|2dt=N⎦summationdisplay n=1|smn|2=/bardblsm/bardbl2(2.2–46) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 35 /H92781(t) /H92783(t) /H92782(t) (b)(a) FIGURE 2.2–1 Gram-Schmidt orthogonalization of the signal {sm(t),m=1,2,3,4}and the corresponding orthonormal basis. The energy in the mth signal is simply the square of the length of the vector or, equiv- alently, the square of the Euclidean distance from the origin to the point smin the N-dimensional space. Thus, any signal can be represented geometrically as a point in the signal space spanned by the orthonormal functions {φn(t)}. From the orthonormality of the basis it also follows that /angbracketleftsk(t),sl(t)/angbracketright=/angbracketleft sk,sl/angbracketright (2.2–47) This shows that the inner product of two signals is equal to the inner product of the corresponding vectors. Proakis-27466 book September 25, 2007 13:9 36 Digital Communications s(t) /H11001/H11003/H11003 /H11003s1 ss2 sN/H92781(t) /H92782(t) /H9278N(t) (a) (b)/H11003/H11003 /H11003s(t)/H9278* 1(t) /H9278* 2(t) /H9278* N(t)s1 s2 s sN . . . . . . . . . . . . . . . FIGURE 2.2–2 Vector to signal (a), and signal to vector (b) mappings. EXAMPLE 2.2–4. Let us obtain the vector representation of the four signals shown in Figure 2.2–1(a) by using the orthonormal set of functions in Figure 2.2–1(b). Sincethe dimensionality of the signal space is N=3, each signal is described by three components. The signal s 1(t) is characterized by the vector s1=(√ 2,0,0)t. Similarly, the signals s2(t),s3(t), and s4(t) are characterized by the vectors s2=(0,√ 2,0)t, s3=(√ 2,0,1)t, and s4=(−√ 2,0,1)t, respectively. These vectors are shown in Figure 2.2–3. Their lengths are /bardbls1/bardbl=√ 2,/bardbls2/bardbl=√ 2,/bardbls3/bardbl=√ 3, and /bardbls4/bardbl=√ 3, and the corresponding signal energies are Ek=/bardblsk/bardbl2,k=1,2,3,4. We have demonstrated that a set of Mﬁnite energy waveforms {sm(t)}can be rep- resented by a weighted linear combination of orthonormal functions {φn(t)}of dimen- sionality N≤M. The functions {φn(t)}are obtained by applying the Gram-Schmidt orthogonalization procedure on {sm(t)}. It should be emphasized, however, that the functions {φn(t)}obtained from the Gram-Schmidt procedure are not unique. If we Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 37 /H92782 /H92783/H92781FIGURE 2.2–3 The four signal vectors represented as points inthree-dimensional space. alter the order in which the orthogonalization of the signals {sm(t)}is performed, the orthonormal waveforms will be different and the corresponding vector representationof the signals {s m(t)}will depend on the choice of the orthonormal functions {φn(t)}. Nevertheless, the dimensionality of the signal space Nwill not change, and the vectors {sm}will retain their geometric conﬁguration; i.e., their lengths and their inner products will be invariant to the choice of the orthonormal functions {φn(t)}. EXAMPLE 2.2–5. An alternative set of orthonormal functions for the four signals in Figure 2.2–1(a) is illustrated in Figure 2.2–4(a). By using these functions to expand{s n(t)}, we obtain the corresponding vectors s1=(1,1,0)t,s2=(1,−1,0)t,s3= (1,1,−1)t, and s4=(−1,−1,−1)t, which are shown in Figure 2.2–4(b). Note that the vector lengths are identical to those obtained from the orthonormal functions {φn(t)}. /H92741(t) /H92742(t) /H92743(t) /H92742 /H92741 /H92743(a) (b) FIGURE 2.2–4 An alternative set of orthonormal functions for the four signals in Figure 2.2–1(a) and thecorresponding signal points. Proakis-27466 book September 25, 2007 13:9 38 Digital Communications Bandpass and Lowpass Orthonormal Basis Let us consider the case in which the signal waveforms are bandpass and represented as sm(t)=Re⎦bracketleftbigsml(t)ej2πf0t⎦bracketrightbig, m=1,2,..., M (2.2–48) where {sml(t)}denotes the lowpass equivalent signals. Recall from Section 2.1–1 that if two lowpass equivalent signals are orthogonal, the corresponding bandpass signals areorthogonal too. Therefore, if {φ nl(t),n=1,..., N}constitutes an orthonormal basis for the set of lowpass signals {sml(t)}, then the set {φn(t),n=1,..., N}where φn(t)=√ 2R e⎦bracketleftbigφnl(t)ej2πf0t⎦bracketrightbig(2.2–49) is a set of orthonormal signals, where√ 2 is a normalization factor to make sure each φn(t) has unit energy. However, this set is not necessarily an orthonormal basis for expansion of {sm(t),m=1,..., M}. In other words, there is no guarantee that this set is a complete basis for expansion of the set of signals {sm(t),m=1,..., M}. Here our goal is to see how an orthonormal basis for representation of bandpass signals can beobtained from an orthonormal basis used for representation of the lowpass equivalentsof the bandpass signals. Since we have s ml(t)=N⎦summationdisplay n=1smlnφnl(t), m=1,..., M (2.2–50) where smln=/angbracketleftsml(t),φnl(t)/angbracketright, m=1,..., M, n=1,..., N (2.2–51) from Equations 2.2–48 and 2.2–50 we can write sm(t)=Re⎦bracketleftBigg⎦parenleftBiggN⎦summationdisplay n=1smlnφnl(t)⎦parenrightBigg ej2πf0t⎦bracketrightBigg , m=1,..., M (2.2–52) or sm(t)=Re⎦bracketleftBiggN⎦summationdisplay n=1smlnφnl(t)⎦bracketrightBigg cos 2πf0t−Im⎦bracketleftBiggN⎦summationdisplay n=1smlnφnl(t)⎦bracketrightBigg sin 2πf0t(2.2–53) In Problem 2.6 we will see that when an orthonormal set of signals {φnl(t),n= 1,..., N}constitutes an N-dimensional complex basis for representation of {sml(t), m=1,..., M}, then the set {φn(t),⎦tildewideφn(t),n=1,..., N}, where φn(t)=√ 2R e⎦bracketleftbigφnl(t)ej2πf0t⎦bracketrightbig=√ 2φni(t) cos 2 πf0t−√ 2φnq(t) sin 2πf0t ⎦tildewideφn(t)=−√ 2I m⎦bracketleftbigφnl(t)ej2πf0t⎦bracketrightbig=−√ 2φni(t) sin 2πf0t−√ 2φnq(t) cos 2 πf0t (2.2–54) constitutes a 2 N-dimensional orthonormal basis that is sufﬁcient for representation of Mbandpass signals sm(t)=Re⎦bracketleftbigsml(t)ej2πf0t⎦bracketrightbig, m=1,..., M (2.2–55) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 39 In some cases not all basis functions in the set of basis given by Equation 2.2–54 are necessary, and only a subset of them would be sufﬁcient to expand the bandpass signals.In Problem 2.7 we will further show that ⎦tildewideφ(t)=−⎦hatwideφ(t) (2.2–56) where ⎦hatwideφ(t) denotes the Hilbert transform of φ(t). From Equation 2.2–52 we have sm(t)=Re⎦bracketleftBigg⎦parenleftBiggN⎦summationdisplay n=1smlnφnl(t)⎦parenrightBigg ej2πf0t⎦bracketrightBigg =N⎦summationdisplay n=1Re⎦bracketleftbig(smlnφnl(t))ej2πf0t⎦bracketrightbig =N⎦summationdisplay n=1⎦bracketleftBigg s(r) mln√ 2φn(t)+s(i) mln√ 2⎦tildewideφn(t)⎦bracketrightBigg(2.2–57) where we have assumed that smln=s(r) mln+js(i) mln. Equations 2.2–54 and 2.2–57 show how a bandpass signal can be expanded in terms of the basis used for expansion of itslowpass equivalent. In general, lowpass signals can be represented by an N-dimensional complex vector, and the corresponding bandpass signal can be represented by 2 N- dimensional real vectors. If the complex vector s ml=(sml1,sml2,..., smlN)t is a vector representation for the lowpass signal sml(t) using the lowpass basis {φnl(t), n=1,..., N}, then the vector sm=⎦parenleftBigg s(r) ml1√ 2,s(r) ml2√ 2,...,s(r) mlN√ 2,s(i) ml1√ 2,s(i) ml2√ 2,...,s(i) mlN√ 2⎦parenrightBiggt (2.2–58) is a vector representation of the bandpass signal sm(t)=Re⎦bracketleftbigsml(t)ej2πf0t⎦bracketrightbig when the bandpass basis {φn(t),⎦tildewideφn(t),n=1,..., N}given by Equations 2.2–54 and 2.2–57 is used. EXAMPLE 2.2–6. Let us assume Mbandpass signals are deﬁned by sm(t)=Re⎦bracketleftbig Amg(t)ej2πf0t⎦bracketrightbig (2.2–59) where Am’s are arbitrary complex numbers and g(t) is a real lowpass signal with energy Eg. The lowpass equivalent signals are given by sml(t)=Amg(t) and therefore the unit-energy signal φ(t) deﬁned by φ(t)=g(t)⎦radicalbig Eg is sufﬁcient to expand all sml(t)’s. Proakis-27466 book September 25, 2007 13:9 40 Digital Communications We have sml(t)=Am⎦radicalbig Egφ(t) thus, corresponding to each sml(t) we have a single complex scalar Am⎦radicalbig Eg=⎦parenleftbig A(r) m+jA(i) m⎦parenrightbig⎦radicalbig Eg; i.e., the lowpass signals constitute one complex dimension (or, equivalently, two real dimensions). From Equation 2.2–54 we conclude that φ(t)=⎦radicalBigg 2 Egg(t) cos 2π f0t ⎦tildewideφ(t)=−⎦radicalBigg 2 Egg(t) sin 2π f0t can be used as a basis for expansion of the bandpass signals. Using this basis and Equation 2.2–57, we have sm(t)=A(r) m⎦radicalbigg Eg 2φ(t)+A(i) m⎦radicalbigg Eg 2⎦tildewideφ(t) =A(r) mg(t) cos 2π f0t−A(i) mg(t) sin 2π f0t which agrees with the straightforward expansion of Equation 2.2–59. Note that in the special case where all Am’s are real, φ(t) is sufﬁcient to represent the bandpass signals and⎦tildewideφ(t) is not necessary. 2.3 SOME USEFUL RANDOM V ARIABLES In subsequent chapters, we shall encounter several different types of random variables. In this section we list these frequently encountered random variables, their probabilitydensity functions (PDFs), their cumulative distribution functions (CDFs), and theirmoments. Our main emphasis will be on the Gaussian random variable and manyrandom variables that are derived from the Gaussian random variable. The Bernoulli Random Variable The Bernoulli random variable is a discrete binary-valued random variable taking values1 and 0 with probabilities pand 1 −p, respectively. Therefore the probability mass function (PMF) for this random variable is given by P[X=1]=p P[X=0]=1−p (2.3–1) The mean and variance of this random variable are given by E[X]=p VA R [X]=p(1−p)(2.3–2) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 41 The Binomial Random Variable The binomial random variable models the sum of nindependent Bernoulli random variables with common parameter p. The PMF of this random variable is given by P[X=k]=⎦parenleftBigg n k⎦parenrightBigg pk(1−p)n−k,k=0,1,..., n (2.3–3) For this random variable we have E[X]=np VA R [X]=np(1−p)(2.3–4) This random variable models, for instance, the number of errors when nbits are trans- mitted over a communication channel and the probability of error for each bit is p. The Uniform Random Variable The uniform random variable is a continuous random variable with PDF p(x)=⎦braceleftBigg 1 b−aa≤x≤b 0 otherwise(2.3–5) where b>aand the interval [ a,b] is the range of the random variable. Here we have E[X]=b−a 2(2.3–6) VA R [X]=(b−a)2 12(2.3–7) The Gaussian (Normal) Random Variable The Gaussian random variable is described in terms of two parameters m∈Rand σ> 0 by the PDF p(x)=1√ 2πσ2e−(x−m)2 2σ2 (2.3–8) We usually use the shorthand form N(m,σ2) to denote the PDF of Gaussian random variables and write X∼N(m,σ2). For this random variable E[X]=m VA R [X]=σ2(2.3–9) A Gaussian random variable with m=0 and σ=1 is called a standard normal .A function closely related to the Gaussian random variable is the Qfunction deﬁned as Q(x)=P[N(0,1)>x]=1√ 2π⎦integraldisplay∞ xe−t2 2dt (2.3–10) Proakis-27466 book September 25, 2007 13:9 42 Digital Communications (a) (b) FIGURE 2.3–1 PDF and CDF of a Gaussian random variable. The CDF of a Gaussian random variable is given by F(x)=⎦integraldisplayx −∞1√ 2πσ2e−(t−m)2 2σ2dt =1−⎦integraldisplay∞ x1√ 2πσ2e−(t−m)2 2σ2dt =1−⎦integraldisplay∞ x−m σ1√ 2πe−u2 2du =1−Q⎦parenleftbiggx−m σ⎦parenrightbigg(2.3–11) where we have introduced the change of variable u=(t−m)/σ. The PDF and the CDF of a Gaussian random variable are shown in Figure 2.3–1. In general if X∼N(m,σ2), then P[X>α]=Q⎦parenleftbiggα−m σ⎦parenrightbigg P[X<α]=Q⎦parenleftbiggm−α σ⎦parenrightbigg (2.3–12) Following are some of the important properties of the Qfunction: Q(0)=1 2Q(∞)=0 (2.3–13) Q(−∞)=1 Q(−x)=1−Q(x) (2.3–14) Some useful bounds for the Qfunction for x>0 are Q(x)≤1 2e−x2 2 Q(x)<1 x√ 2πe−x2 2 Q(x)>x (1+x2)√ 2πe−x2 2(2.3–15) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 43 0 0.5 1 1.5 2 2.5 300.10.20.30.40.50.61 x√2/H92662e/H11002x2 1 22e/H11002x2 Q(x) x (1 /H11001 x2)√2/H92662e/H11002x2 FIGURE 2.3–2 Plot of Q(x) and its upper and lower bounds. From the last two bounds we conclude that for large xwe have Q(x)≈1 x√ 2πe−x2 2 (2.3–16) A plot of the Qfunction bounds is given in Figure 2.3–2. Tables 2.3–1 and 2.3–2 give values of the Qfunction. TABLE 2.3–1 Table of QFunction Values xQ (x) xQ (x) xQ (x) xQ (x) 0 0.500000 1.8 0.035930 3.6 0.000159 5.4 3.3320 ×10−8 0.1 0.460170 1.9 0.028717 3.7 0.000108 5.5 1.8990 ×10−8 0.2 0.420740 2 0.022750 3.8 7.2348 ×10−55.6 1.0718 ×10−8 0.3 0.382090 2.1 0.017864 3.9 4.8096 ×10−55.7 5.9904 ×10−9 0.4 0.344580 2.2 0.013903 4 3.1671 ×10−55.8 3.3157 ×10−9 0.5 0.308540 2.3 0.010724 4.1 2.0658 ×10−55.9 1.8175 ×10−9 0.6 0.274250 2.4 0.008198 4.2 1.3346 ×10−56 9.8659 ×10−10 0.7 0.241960 2.5 0.006210 4.3 8.5399 ×10−66.1 5.3034 ×10−10 0.8 0.211860 2.6 0.004661 4.4 5.4125 ×10−66.2 2.8232 ×10−10 0.9 0.184060 2.7 0.003467 4.5 3.3977 ×10−66.3 1.4882 ×10−10 1 0.158660 2.8 0.002555 4.6 2.1125 ×10−66.4 7.7689 ×10−11 1.1 0.135670 2.9 0.001866 4.7 1.3008 ×10−66.5 4.0160 ×10−11 1.2 0.115070 3 0.001350 4.8 7.9333 ×10−76.6 2.0558 ×10−11 1.3 0.096800 3.1 0.000968 4.9 4.7918 ×10−76.7 1.0421 ×10−11 1.4 0.080757 3.2 0.000687 5 2.8665 ×10−76.8 5.2309 ×10−12 1.5 0.066807 3.3 0.000483 5.1 1.6983 ×10−76.9 2.6001 ×10−12 1.6 0.054799 3.4 0.000337 5.2 9.9644 ×10−87 1.2799 ×10−12 1.7 0.044565 3.5 0.000233 5.3 5.7901 ×10−87.1 6.2378 ×10−13 Proakis-27466 book September 25, 2007 13:9 44 Digital Communications TABLE 2.3–2 Selected QFunction Values Q(x) x 10−11.2816 10−22.3263 10−33.0902 10−43.7190 10−54.2649 10−64.7534 10−75.1993 0.5×10−54.4172 0.25×10−54.5648 0.667×10−54.3545 Another function closely related to the Qfunction is the complementary error function , deﬁned as erfc( x)=2√π⎦integraldisplay∞ xe−t2dt (2.3–17) The complementary error function is related to the Qfunction as follows: Q(x)=1 2erfc⎦parenleftbiggx√ 2⎦parenrightbigg erfc( x)=2Q(√ 2x)(2.3–18) The characteristic function†of a Gaussian random variable is given by /Phi1X(ω)=ejωm−1 2ω2σ2(2.3–19) Problem 2.21 shows that for an N(m,σ2) random variable we have E⎦bracketleftbig(X−m)n⎦bracketrightbig=⎦braceleftBigg 1×3×5×···× (2k−1)σ2k=(2k)!σ2k 2kk!forn=2k 0 for n=2k+1 (2.3–20) from which we can obtain moments of the Gaussian random variable. The sum of nindependent Gaussian random variables is a Gaussian random variable whose mean and variance are the sum of the means and the sum of the variances of therandom variables, respectively. †Recall that for any random variable X, the characteristic function is deﬁned by /Phi1X(ω)=E[ejωX]. Themoment generating function (MGF) is deﬁned by /Theta1X(t)=E[etX]. Obviously, /Theta1(t)=/Phi1(−jt) and /Phi1(ω)=/Theta1(jω). Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 45 The Chi-Square ( χ2) Random Variable If{Xi,i=1,..., n}are iid (independent and identically distributed) zero-mean Gaussian random variables with common variance σ2and we deﬁne X=n⎦summationdisplay i=1X2 i then Xis aχ2random variable with n degrees of freedom . The PDF of this random variable is given by p(x)=⎦braceleftBigg1 2n/2/Gamma1(n 2)σnxn 2−1e−x 2σ2 x>0 0 otherwise(2.3–21) where /Gamma1(x)i st h e gamma function deﬁned by /Gamma1(x)=⎦integraldisplay∞ 0tx−1e−tdt, (2.3–22) The gamma function has simple poles at x=0,−1,−2,−3,... and satisﬁes the following properties. The gamma function can be thought of as a generalization of thenotion of factorial. /Gamma1(x+1)=x/Gamma1(x), /Gamma1(1)=1 /Gamma1⎦parenleftbigg1 2⎦parenrightbigg =√π /Gamma1⎦parenleftbiggn 2+1⎦parenrightbigg =⎦braceleftBigg⎦parenleftbign 2⎦parenrightbig! neven and positive√πn(n−2)(n−4)...3×1 2n+1 2nodd and positive(2.3–23) When nis even, i.e., n=2m, the CDF of the χ2random variable with ndegrees of freedom has a closed form given by F(x)=⎧ ⎪⎨ ⎪⎩1−e−x 2σ2m−1⎦summationdisplay k=01 k!⎦parenleftbiggx 2σ2⎦parenrightbiggk x>0 0 otherwise(2.3–24) The mean and variance of a χ2random variable with ndegrees of freedom are given by E[X]=nσ2 VA R [X]=2nσ4(2.3–25) The characteristic function for a χ2random variable with ndegrees of freedom is given by /Phi1(ω)=⎦parenleftbigg1 1−2jωσ2⎦parenrightbiggn 2 (2.3–26) Proakis-27466 book September 25, 2007 13:9 46 Digital Communications The special case of a χ2random variable with two degrees of freedom is of particular interest. In this case the PDF is given by p(x)=⎦braceleftBigg 1 2σ2e−x 2σ2 x>0 0 otherwise(2.3–27) This is the PDF of an exponential random variable with mean equal to 2 σ2. Theχ2random variable is a special case of a gamma random variable . A gamma random variable is deﬁned by a PDF of the form p(x)=⎦braceleftBiggλ(λx)α−1e−λx /Gamma1(α)x≥0 0 otherwise(2.3–28) where λ,α > 0. Aχ2random variable is a gamma random variable with λ=1 2σ2and α=n 2. Plots of the χ2random variable with ndegrees of freedom for different values of nare shown in Figure 2.3–3. The Noncentral Chi-Square ( χ2) Random Variable Thenoncentral χ2random variable with n degrees of freedom is deﬁned similarly to a χ2random variable in which Xi’s are independent Gaussians with common variance σ2but with different means denoted by mi. This random variable has a PDF of the form p(x)=⎦braceleftBigg 1 2σ2⎦parenleftbigx s2⎦parenrightbign−2 4e−s2+x 2σ2In 2−1⎦parenleftbigs σ2√x⎦parenrightbigx>0 0 otherwise(2.3–29) 0 1 2 3 4 5 600.10.20.30.40.50.60.70.80.91 n /H11005 1 n /H11005 2 n /H11005 3 n /H11005 4n /H11005 5 n /H11005 6 FIGURE 2.3–3 The PDF of the χ2random variable for different values of n. All plots are shown for σ=1. Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 47 where sis deﬁned as s=⎦radicaltp⎦radicalvertex⎦radicalvertex⎦radicalbtn⎦summationdisplay i=1m2 i (2.3–30) andIα(x)i st h e modiﬁed Bessel function of the ﬁrst kind and order αgiven by Iα(x)=∞⎦summationdisplay k=0(x/2)α+2k k!/Gamma1(α+k+1),x≥0 (2.3–31) where /Gamma1(x) is the gamma function deﬁned by Equation 2.3–22. The function I0(x) can be written as I0(x)=∞⎦summationdisplay k=0⎦parenleftBigg xk 2kk!⎦parenrightBigg2 (2.3–32) and for x>1 can be approximated by I0(x)≈ex √ 2πx(2.3–33) Two other expressions for I0(x), which are used frequently, are I0(x)=1 π⎦integraldisplayπ 0e±xcosφdφ I0(x)=1 2π⎦integraldisplay2π 0excosφdφ(2.3–34) The CDF of this random variable, when n=2m, can be written in the form F(x)=⎦braceleftBigg 1−Qm⎦parenleftBig s σ,√x σ⎦parenrightBig x>0 0 otherwise(2.3–35) where Qm(a,b)i st h e generalized Marcum Q function and is deﬁned as Qm(a,b)=⎦integraldisplay∞ bx⎦parenleftbiggx a⎦parenrightbiggm−1 e−(x2+a2)/2Im−1(ax)dx =Q1(a,b)+e−(a2+b2)/2m−1⎦summationdisplay k=1⎦parenleftbiggb a⎦parenrightbiggk Ik(ab)(2.3–36) In Equation 2.3–36, Q1(a,b)i st h e Marcum Q function deﬁned as Q1(a,b)=⎦integraldisplay∞ bxe−a2+x2 2I0(ax)dx (2.3–37) or Q1(a,b)=e−a2+b2 2∞⎦summationdisplay k=0⎦parenleftbigga b⎦parenrightbiggk Ik(ab), b≥a>0 (2.3–38) Proakis-27466 book September 25, 2007 13:9 48 Digital Communications This function satisﬁes the following properties: Q1(x,0)=1 Q1(0,x)=e−x2 2 Q1(a,b)≈Q(b−a) for b/greatermuch1 and b/greatermuchb−a(2.3–39) For a noncentral χ2random variable, the mean and variance are given by E[X]=nσ2+s2 VA R [X]=2nσ4+4σ2s2(2.3–40) and the characteristic function is given by /Phi1(ω)=⎦parenleftbigg1 1−2jωσ2⎦parenrightbiggn 2 ejωs2 1−2jωσ2(2.3–41) The Rayleigh Random Variable IfX1and X2are two iid Gaussian random variables each distributed according to N(0,σ2), then X=⎦radicalBig X2 1+X2 2 (2.3–42) is aRayleigh random variable . From our discussion of the χ2random variables, it is readily seen that a Rayleigh random variable is the square root of a χ2random variable with two degrees of freedom. We can also conclude that the Rayleigh random variableis the square root of an exponential random variable as given by Equation 2.3–27. ThePDF of a Rayleigh random variable is given by p(x)=⎦braceleftBigg x σ2e−x2 2σ2 x>0 0 otherwise(2.3–43) and its mean and variance are E[X]=σ⎦radicalbiggπ 2 VA R [X]=⎦parenleftbigg 2−π 2⎦parenrightbigg σ2(2.3–44) In general, the nth moment of a Rayleigh random variable is given by E⎦bracketleftbigXk⎦bracketrightbig=(2σ2)k/2/Gamma1⎦parenleftbiggk 2+1⎦parenrightbigg (2.3–45) and its characteristic function is given by /Phi1X(ω)=1F1⎦parenleftbigg 1,1 2;−1 2ω2σ2⎦parenrightbigg +j⎦radicalbiggπ 2ωσe−ω2σ2 2 (2.3–46) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 49 where 1F1(a,b;x)is the conﬂuent hypergeometric function deﬁned by 1F1(a,b;x)=∞⎦summationdisplay k=0/Gamma1(a+k)/Gamma1(b)xk /Gamma1(a)/Gamma1(b+k)k!, b/negationslash=0,−1,−2,... (2.3–47) The function 1F1(a,b;x)can also be written as the integral 1F1(a,b;x)=/Gamma1(b) /Gamma1(b−a)/Gamma1(a)⎦integraldisplay1 0extta−1(1−t)b−a−1dt (2.3–48) In Beaulieu (1990), it is shown that 1F1⎦parenleftbigg 1,1 2;−x⎦parenrightbigg =−e−x∞⎦summationdisplay k=0xk (2k−1)k!(2.3–49) The CDF of a Rayleigh random variable can be easily found by integrating the PDF. The result is F(x)=⎦braceleftBigg 1−e−x2 2σ2 x>0 0 otherwise(2.3–50) The PDF of a Rayleigh random variable is plotted in Figure 2.3–4. A generalized version of the Rayleigh random variable is obtained when we have niid zero-mean Gaussian random variables {Xi,1≤i≤n}where each Xihas an N(0,σ2) distribution. In this case X=⎦radicaltp⎦radicalvertex⎦radicalvertex⎦radicalbtn⎦summationdisplay i=1X2 i (2.3–51) has a generalized Rayleigh distribution . The PDF for this random variable is given by p(x)=⎧ ⎨ ⎩xn−1 2n−2 2σn/Gamma1(n 2)e−x2 2σ2 x≥0 0 otherwise(2.3–52) p(x) x/H92681 /H11021 /H9268 /H9268 /H92682 /H11022 /H9268FIGURE 2.3–4 The PDF of the Rayleigh random variablefor three different values of σ. Proakis-27466 book September 25, 2007 13:9 50 Digital Communications For the generalized Rayleigh, and with n=2m, the CDF is given by F(x)=⎧ ⎨ ⎩1−e−x2 2σ2⎦summationtextm−1 k=01 k!⎦parenleftBig x2 2σ2⎦parenrightBigk x≥0 0 otherwise(2.3–53) Thekth moment of a generalized Rayleigh for any integer value of n(even or odd) is given by E⎦bracketleftbigXk⎦bracketrightbig=(2σ2)k 2/Gamma1⎦parenleftbign+k 2⎦parenrightbig /Gamma1⎦parenleftbign 2⎦parenrightbig (2.3–54) The Ricean Random Variable IfX1andX2are two independent Gaussian random variables distributed according to N(m1,σ2) andN(m2,σ2) (i.e., the variances are equal and the means may be different), then X=⎦radicalBig X2 1+X2 2 (2.3–55) is a Ricean random variable with PDF p(x)=⎦braceleftBigg x σ2I0⎦parenleftbigsx σ2⎦parenrightbige−x2+s2 2σ2 x>0 0 otherwise(2.3–56) where s=⎦radicalBig m2 1+m2 2andI0(x) is given by Equation 2.3–32. It is clear that a Ricean random variable is the square root of a noncentral χ2random variable with two degrees of freedom. It is readily seen that for s=0, the Ricean random variable reduces to a Rayleigh random variable. For large sthe Ricean random variable can be well approximated by a Gaussian random variable. The CDF of a Ricean random variable can be expressed as F(x)=⎦braceleftBigg 1−Q1⎦parenleftbigs σ,x σ⎦parenrightbigx>0 0 otherwise(2.3–57) where Q1(a,b) is deﬁned by Equations 2.3–37 and 2.3–38. The ﬁrst two moments of the Ricean random variable are given by E[X]=σ⎦radicalbiggπ 21F1⎦parenleftBigg −1 2,1,−s2 2σ2⎦parenrightBigg =σ⎦radicalbiggπ 2e−K 2⎦bracketleftbigg (1+K)I0⎦parenleftbiggK 2⎦parenrightbigg +KI1⎦parenleftbiggK 2⎦parenrightbigg⎦bracketrightbigg E⎦bracketleftbigX2⎦bracketrightbig=2σ2+s2(2.3–58) where Kis the Rice factor deﬁned in Equation 2.3–60. Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 51 In general, the kth moment of this random variable is given by E⎦bracketleftbigXk⎦bracketrightbig=(2σ2)k 2/Gamma1⎦parenleftbigg 1+k 2⎦parenrightbigg 1F1⎦parenleftBigg −k 2,1;−s2 2σ2⎦parenrightBigg (2.3–59) Another form of the Ricean density function is obtained by deﬁning the Rice factor Kas K=s2 2σ2(2.3–60) If we deﬁne A=s2+2σ2, the Ricean PDF can be written as p(x)=⎧ ⎨ ⎩2(K+1) Axe−K+1 A(x2+AK K+1)I0⎦parenleftbigg 2x⎦radicalBig K(K+1) A⎦parenrightbigg x≥0 0 otherwise(2.3–61) For the normalized case when A=1 (or, equivalently, when E⎦bracketleftbigX2⎦bracketrightbig=s2+2σ2=1) this reduces to p(x)=⎦braceleftBigg 2(K+1)xe−(K+1)(x2+K K+1)I0⎦parenleftbig2x√K(K+1)⎦parenrightbigx≥0 0 otherwise(2.3–62) A plot of the PDF of a Ricean random variable for different values of Kis shown in Figure 2.3–5. Similar to the Rayleigh random variable, a generalized Ricean random variable can be deﬁned as X=⎦radicaltp⎦radicalvertex⎦radicalvertex⎦radicalbtn⎦summationdisplay i=1X2 i (2.3–63) 1.41.8 11.6K /H11005 10 K /H11005 1 K /H11005 0.12 1.2 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 2 2.5 3 FIGURE 2.3–5 The Ricean PDF for different values of K. For small Kthis random variable reduces to a Rayleigh random variable, and for large Kit is well approximated by a Gaussian random variable. Proakis-27466 book September 25, 2007 13:9 52 Digital Communications where Xi’s are independent Gaussians with mean miand common variance σ2. In this case the PDF is given by p(x)=⎧ ⎨ ⎩xn 2 σ2sn−2 2e−x2+s2 2σ2In 2−1⎦parenleftbigxs σ2⎦parenrightbigx≥0 0 otherwise(2.3–64) and the CDF is given by F(x)=⎦braceleftBigg 1−Qm⎦parenleftbigs σ,x σ⎦parenrightbigx≥0 0 otherwise(2.3–65) where s=⎦radicaltp⎦radicalvertex⎦radicalvertex⎦radicalbtn⎦summationdisplay i=1m2 i Thekth moment of a generalized Ricean is given by E⎦bracketleftbigXk⎦bracketrightbig=(2σ2)k 2e−s2 2σ2/Gamma1⎦parenleftbign+k 2⎦parenrightbig /Gamma1⎦parenleftbign 2⎦parenrightbig1F1⎦parenleftBigg n+k 2,n 2;s2 2σ2⎦parenrightBigg (2.3–66) The Nakagami Random Variable Both the Rayleigh distribution and the Rice distribution are frequently used to describe the statistical ﬂuctuations of signals received from a multipath fading channel. Thesechannel models are considered in Chapters 13 and 14. Another distribution that isfrequently used to characterize the statistics of signals transmitted through multipathfading channels is the Nakagami mdistribution. The PDF for this distribution is given by Nakagami (1960) as p(x)= ⎦braceleftBigg2 /Gamma1(m)⎦parenleftbigm /Omega1⎦parenrightbigmx2m−1e−mx2//Omega1x>0 0 otherwise(2.3–67) where /Omega1is deﬁned as /Omega1=E⎦bracketleftbigX2⎦bracketrightbig(2.3–68) and the parameter mis deﬁned as the ratio of moments, called the fading ﬁgure , m=/Omega12 E⎦bracketleftBig⎦parenleftbigX2−/Omega1⎦parenrightbig2⎦bracketrightBig, m≥1 2(2.3–69) A normalized version of Equation 2.3–67 may be obtained by deﬁning another random variable Y=X/√ /Omega1(see Problem 2.42). The nth moment of Xis E⎦bracketleftbigXn⎦bracketrightbig=/Gamma1⎦parenleftbigm+n 2⎦parenrightbig /Gamma1(m)⎦parenleftbigg/Omega1 m⎦parenrightbiggn/2 (2.3–70) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 53 The mean and the variance for this random variable are given by E[X]=/Gamma1⎦parenleftbigm+1 2⎦parenrightbig /Gamma1(m)⎦parenleftbigg/Omega1 m⎦parenrightbigg1/2 VA R [X]=/Omega1⎛ ⎝1−1 m⎦parenleftBigg /Gamma1⎦parenleftbigm+1 2⎦parenrightbig /Gamma1(m)⎦parenrightBigg2⎞ ⎠(2.3–71) By setting m=1, we observe that Equation 2.3–67 reduces to a Rayleigh PDF. For values of min the range1 2≤m≤1, we obtain PDFs that have larger tails than a Rayleigh-distributed random variable. For values of m>1, the tail of the PDF decays faster than that of the Rayleigh. Figure 2.3–6 illustrates the Nakagami PDF for differentvalues of m. FIGURE 2.3–6 The PDF for the Nakagami mdistribution, shown with /Omega1=1.mis the fading ﬁgure. Proakis-27466 book September 25, 2007 13:9 54 Digital Communications The Lognormal Random Variable Suppose that a random variable Yis normally distributed with mean mand variance σ2. Let us deﬁne a new random variable Xthat is related to Ythrough the transformation Y=lnX(orX=eY). Then the PDF of Xis p(x)=⎦braceleftBigg1√ 2πσ2xe−(lnx−m)2/2σ2x≥0 0 otherwise(2.3–72) For this random variable E[X]=em+σ2 2 VA R [X]=e2m+σ2⎦parenleftBig eσ2−1⎦parenrightBig (2.3–73) The lognormal distribution is suitable for modeling the effect of shadowing of the signal due to large obstructions, such as tall buildings, in mobile radio communications.Examples of the lognormal PDF are shown in Figure 2.3–7. Jointly Gaussian Random Variables Ann×1 column random vector Xwith components {X i,1≤i≤n}is called a Gaussian vector , and its components are called jointly Gaussian random variables or 0.5m /H11005 0 m /H11005 1 m /H11005 2 m /H11005 30.7 0.6 0.4 0.30.20.1 0 0 5 10 15 FIGURE 2.3–7 Lognormal PDF with σ=1 for different values of m. Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 55 multivariate Gaussian random variables if the joint PDF of Xi’s can be written as p(x)=1 (2π)n/2(detC)1/2e−1 2(x−m)tC−1(x−m)(2.3–74) where mandCare the mean vector and covariance matrix, respectively, of Xand are given by m=E[X] C=E⎦bracketleftbig(X−m)(X−m)t⎦bracketrightbig (2.3–75) From this deﬁnition it is clear that Cij=COV⎦bracketleftbigXi,Xj⎦bracketrightbig(2.3–76) and therefore Cis a symmetric matrix. From elementary probability it is also well known that Cis nonnegative deﬁnite. In the special case of n=2, we have m=⎦bracketleftbiggm1 m2⎦bracketrightbigg C=⎦bracketleftbiggσ2 1ρσ1σ2 ρσ1σ2σ2 2⎦bracketrightbigg (2.3–77) where ρ=COV [X1,X2] σ1σ2 is the correlation coefﬁcient of the two random variables. In this case the PDF reduces to p(x1,x2)=1 2πσ1σ2⎦radicalbig 1−ρ2e−⎦parenleftbigx1−m1σ1⎦parenrightbig2 +⎦parenleftbigx2−m2σ2⎦parenrightbig2 −2ρ⎦parenleftbigx1−m1σ1⎦parenrightbig⎦parenleftbigx2−m2σ2⎦parenrightbig 2(1−ρ2) (2.3–78) where m1,m2,σ2 1and,σ2 2are means and variances of the two random variables and ρ is their correlation coefﬁcient. Note that in the special case when ρ=0 (i.e., when the two random variables are uncorrelated), we have p(x1,x2)=N⎦parenleftbigm1,σ2 1⎦parenrightbig×N⎦parenleftbigm2,σ2 2⎦parenrightbig This means that the two random variables are independent, and therefore for this case independence and uncorrelatedness are equivalent. This property is true for generaljointly Gaussian random variables. Another important property of jointly Gaussian random variables is that linear combinations of jointly Gaussian random variables are also jointly Gaussian. In otherwords, if Xis a Gaussian vector, the random vector Y=AX, where the invertible matrix Arepresents a linear transformation, is also a Gaussian vector whose mean and Proakis-27466 book September 25, 2007 13:9 56 Digital Communications covariance matrix are given by mY=Am X CY=AC XAt(2.3–79) This property is developed in Problem 2.23. In summary, jointly Gaussian random variables have the following important properties: 1. For jointly Gaussian random variables, uncorrelated is equivalent to independent. 2. Linear combinations of jointly Gaussian random variables are themselves jointly Gaussian. 3. The random variables in any subset of jointly Gaussian random variables are jointly Gaussian, and any subset of random variables conditioned on random variables in any other subset is also jointly Gaussian (all joint subsets and all conditional subsetsare Gaussian). We also emphasize that any set of independent Gaussian random variables is jointly Gaussian, but this is not necessarily true for a set of dependent Gaussian randomvariables. Table 2.3–3 summarizes some of the properties of the most important random variables. 2.4 BOUNDS ON TAIL PROBABILITIES Performance analysis of communication systems requires computation of error proba-bilities of these systems. In many cases, as we will observe in the following chapters,the error probability of a communication system is expressed in terms of the probabilitythat a random variable exceeds a certain value, i.e., in the form of P [X>α]. Unfortu- nately, in many cases these probabilities cannot be expressed in closed form. In suchcases we are interested in ﬁnding upper bounds on these tail probabilities. These upperbounds are of the form P [X>α]≤β. In this section we describe different methods for providing and tightening such bounds. The Markov Inequality The Markov inequality gives an upper bound on the tail probability of nonnegativerandom variables. Let us assume that Xis a nonnegative random variable, i.e., p(x)=0 for all x<0, and assume α> 0 is an arbitrary positive real number. The Markov inequality states that P[X≥α]≤E[X] α(2.4–1) Proakis-27466 book September 25, 2007 13:9 TABLE 2.3–3 Properties of Important Random Variables RV (Parameters) PDF or PMF E [X] VA R [X] /Phi1X(ω)=E⎦bracketleftbig ejωX⎦bracketrightbig Bernoulli ( p) P(X=1)=1−P(X=0)=pp p (1−p) pejω+(1−p) 0≤p≤1 Binomial ( n,p) P(X=k)=⎦parenleftbign k⎦parenrightbig pk(1−p)n−knp np (1−p)⎦parenleftbig pejω+(1−p)⎦parenrightbign 0≤k≤n,0≤p≤1 Uniform ( a,b)1 b−a,a≤x≤ba+b 2(b−a)2 12ejωb−ejωa jω(b−a) Exponential ( λ) λe−λx,λ > 0,x≥01 λ1 λ2λ λ−jω Gaussian ( m,σ2)1√ 2πσ2e−(x−m)2 2σ2 m σ2ejωm−ω2σ2 2 σ> 0 Gamma ( λ,α)λ(λx)α−1e−λx /Gamma1(α)α λα λ2⎦parenleftbigλ λ−jω⎦parenrightbigα x≥0,λ , α > 0 χ2(n,σ2)1 2n/2/Gamma1(n 2)σnxn 2−1e−x 2σ2 nσ22nσ4⎦parenleftbig1 1−2jωσ2⎦parenrightbign/2 x,σ > 0,n∈N Noncentral1 2σ2⎦parenleftbigx s2⎦parenrightbign−2 4e−s2+x 2σ2In 2−1⎦parenleftbigs σ2√x⎦parenrightbig nσ2+s22nσ4+4σ2s2⎦parenleftbig1 1−2jωσ2⎦parenrightbign/2ejωs2 1−2jwσ2 χ2(n,s,σ2) x,s,σ > 0,n∈N Rayleigh ( σ2)x σ2e−x2/2σ2σ⎦radicalbigπ 2⎦parenleftbig 2−π 2⎦parenrightbig σ21F1⎦parenleftbig 1,1 2;−ω2σ2 2⎦parenrightbig +j⎦radicalbigπ 2ωσe−ω2σ2 2 x,σ > 0 Ricean ( σ2,s)x σ2I0⎦parenleftbigxs σ2⎦parenrightbig e−x2+s2 2σ2 σ⎦radicalbigπ 21F1⎦parenleftbig −1 2,1,−s2 2σ2⎦parenrightbig 2σ2+s2−(E[X])2— x,s,σ > 0 Jointly Gaussian ( m,C)1 (2π)n/2det(C)e−1 2[(x−m)tC−1(x−m)] mC ejmtω−1 2ωtCω Csymmetric and positive deﬁnite (cov. matrix) 57 Proakis-27466 book September 25, 2007 13:9 58 Digital Communications To see this, we observe that E[X]=⎦integraldisplay∞ 0xp(x)dx ≥⎦integraldisplay∞ αxp(x)dx ≥α⎦integraldisplay∞ αxp(x)dx =αP[X≥α](2.4–2) Dividing both sides by αgives the desired inequality. Chernov Bound The Chernov bound is a very tight and useful bound that is obtained from the Markov inequality. Unlike the Markov inequality that is applicable only to nonnegative randomvariables, the Chernov bound can be applied to all random variables. LetXbe an arbitrary random variable, and let δandνbe arbitrary real numbers (ν/negationslash=0). Deﬁne random variable YbyY=e νXand constant αbyα=eνδ. Obviously, Yis a nonnegative random variable and αis a positive real number. Applying the Markov inequality to Yandαyields P⎦bracketleftbigeνX≥eνδ⎦bracketrightbig≤E⎦bracketleftbigeνX⎦bracketrightbig eνδ=E⎦bracketleftbigeν(X−δ)⎦bracketrightbig(2.4–3) The event {eνX≥eνδ}is equivalent to the event {νX≥νδ}which for positive or negative values of νis equivalent to {X≥δ}or{X≤δ}, respectively. Therefore we have P[X≥δ]≤E⎦bracketleftbigeν(X−δ)⎦bracketrightbig,for all ν>0 (2.4–4) P[X≤δ]≤E⎦bracketleftbigeν(X−δ)⎦bracketrightbig,for all ν<0 (2.4–5) Since the two inequalities are valid for all positive and negative values of ν, re- spectively, it makes sense to ﬁnd the values of νthat give the tightest possible bounds. To this end, we differentiate the right hand of the inequalities with respect to νand ﬁnd its root; this is the value of νthat gives the tightest bound. From this point on, we will consider only the ﬁrst inequality. The extension to the second inequality isstraightforward. Let us deﬁne function g(ν) to denote the right side of the inequalities, i.e., g(ν)=E⎦bracketleftbigeν(X−δ)⎦bracketrightbig Differentiating g(ν), we have g/prime(ν)=E⎦bracketleftbig(X−δ)eν(X−δ)⎦bracketrightbig(2.4–6) The second derivative of g(ν)i sg i v e nb y g/prime/prime(ν)=E⎦bracketleftbig(X−δ)2eν(X−δ)⎦bracketrightbig Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 59 It is easily seen that for all ν,w eh a v e g/prime/prime(ν)>0 and hence g(ν) is convex and g/prime(ν)i s an increasing function, and therefore can have only one root. In addition, since g(ν)i s convex, this single root minimizes g(ν) and therefore results in the the tightest bound. Putting g/prime(ν)=0, we ﬁnd the root to be obtained by solving the equation E⎦bracketleftbigXeνX⎦bracketrightbig=δE⎦bracketleftbigeνX⎦bracketrightbig(2.4–7) Equation 2.4–7 has a single root ν∗that gives the tightest bound. The only thing that remains to be checked is to see whether this ν∗satisﬁes the ν∗>0 condition. Since g/prime(ν) is an increasing function, its only root is positive if g/prime(0)<0. From Equation 2.4–6 we have g/prime(0)=E[X]−δ therefore ν∗>0 if and only if δ>E[X]. Summarizing, from Equations 2.4–4 and 2.4–5 we conclude P[X≥δ]≤e−ν∗δE⎦bracketleftBig eν∗X⎦bracketrightBig ,forδ>E[X] (2.4–8) P[X≤δ]≤e−ν∗δE⎦bracketleftBig eν∗X⎦bracketrightBig ,forδ<E[X] (2.4–9) where ν∗is the solution of Equation 2.4–7. Equations 2.4–8 and 2.4–9 are known as Chernov bounds. Finding optimal ν∗by solving Equation 2.4–7 is sometimes difﬁcult. In such cases a numerical approximation or an educated guess gives a suboptimalbound. The Chernov bound can also be given in terms of the moment generatingfunction (MGF) /Theta1 X(ν)=E⎦bracketleftbigeνX⎦bracketrightbigas P[X≥δ]≤e−ν∗δ/Theta1X(ν∗),forδ>E[X] (2.4–10) P[X≤δ]≤e−ν∗δ/Theta1X(ν∗),forδ<E[X] (2.4–11) EXAMPLE 2.4–1. Consider the Laplace PDF given by p(x)=1 2e−|x|(2.4–12) Let us evaluate the upper tail probability P [X≥δ]for some δ>0 from the Chernov bound and compare it with the true tail probability, which is P[X≥δ]=⎦integraldisplay∞ δ1 2e−xdx=1 2e−δ(2.4–13) First note that E [X]=0, and therefore the condition δ> E[X]needed to use the upper tail probability in the Chernov bound is satisﬁed. To solve Equation 2.4–7 for ν∗, we must determine E⎦bracketleftbig XeνX⎦bracketrightbig and E⎦bracketleftbig eνX⎦bracketrightbig . For the PDF in Equation 2.4–12, we ﬁnd that E⎦bracketleftbig XeνX⎦bracketrightbig and E⎦bracketleftbig eνX⎦bracketrightbig converge only if −1<ν< 1, and for this range of values ofνwe have E⎦bracketleftbig XeνX⎦bracketrightbig =2ν (ν+1)2(ν−1)2 E⎦bracketleftbig eνX⎦bracketrightbig =1 (1+ν)(1−ν)(2.4–14) Proakis-27466 book September 25, 2007 13:9 60 Digital Communications Substituting these values into Equation 2.4–7, we obtain the quadratic equation ν2δ+2ν−δ=0 which has the solutions ν∗=−1±√ 1+δ2 δ(2.4–15) Since ν∗must be in the ( −1,+1) interval for E⎦bracketleftbig XeνX⎦bracketrightbig and E⎦bracketleftbig eνX⎦bracketrightbig to converge, the only acceptable solution is ν∗=−1+√ 1+δ2 δ(2.4–16) Finally, we evaluate the upper bound in Equation 2.4–8 by substituting for ν∗from Equation 2.4–16. The result is P[X≥δ]≤δ2 2(−1+√ 1+δ2)e1−√ 1+δ2(2.4–17) Forδ/greatermuch1, Equation 2.4–17 reduces to P(X≥δ)≤δ 2e−δ(2.4–18) We note that the Chernov bound decreases exponentially as δincreases. Consequently, it approximates closely the exact tail probability given by Equation 2.4–13. EXAMPLE 2.4–2. In performance analysis of communication systems over fading chan- nels, we encounter random variables of the form X=d2R2+2RdN (2.4–19) where dis a constant, Ris a Ricean random variable with parameters sandσrepresent- ing channel attenuation due to fading, and Nis a zero-mean Gaussian random variable with varianceN0 2representing channel noise. It is assumed that RandNare indepen- dent random variables. We are interested to apply the Chernov bounding technique toﬁnd an upper bound on P [X<0]. From the Chernov bound given in Equation 2.4–5, we have P[X≤0]≤E⎦bracketleftbig e νX⎦bracketrightbig ,for all ν<0 (2.4–20) To determine E⎦bracketleftbig eνX⎦bracketrightbig , we use the well-known relation E[Y]=E[E[Y|X]] (2.4–21) from elementary probability. We note that conditioned on R,Xis a Gaussian random variable with mean d2R2and variance 2 R2d2N0. Using the relation for the moment generating function of a Gaussian random variable from Table 2.3–3, we have E⎦bracketleftbig eνX|R⎦bracketrightbig =eνd2R2+ν2d2N0R2=eνd2(1+N0ν)R2(2.4–22) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 61 Now noting that R2is a noncentral χ2random variable with two degrees of freedom, and using the characteristic function for this random variable from Table 2.3–3, weobtain E⎦bracketleftbig e νX⎦bracketrightbig =E⎦bracketleftbig E⎦bracketleftbig eνX|R⎦bracketrightbig⎦bracketrightbig =E⎦bracketleftBig eνd2(1+N0ν)R2⎦bracketrightBig =1 1−2νd2(1+N0ν)σ2eνd2(1+N0ν)s2 1−2νd2(1+N0ν)σ2(2.4–23) where we have used Equation 2.4–21. From Equations 2.4–20 and 2.4–23 we conclude that P[X≤0]≤min ν<01 1−2νd2(1+N0ν)σ2eνd2(1+N0ν)s2 1−2νd2(1+N0ν)σ2(2.4–24) It can be easily veriﬁed by differentiation that in the range of interest ( ν<0), the right- hand side is an increasing function of λ=νd2(1+N0ν), and therefore the minimum is achieved when λis minimized. By simple differentiation we can verify that λis minimized for ν=−1 2N0, resulting in P[X≤0]≤1 1+d2 2N0σ2e−d2 4N0s2 1+d2 2N0σ2(2.4–25) If we use Equation 2.3–61 or 2.3–62 for the Ricean random variable, we obtain the following bounds: P[X≤0]≤K+1 K+1+A2d2 4N0e−A2Kd2 4N0 K+1+d2A2 4N0 (2.4–26) and P[X≤0]≤K+1 K+1+d2 4N0e−Kd2 4N0 K+1+d2 4N0 (2.4–27) For the case of Rayleigh fading channels, in which s=0, these relations reduce to P[X≤0]≤1 1+d2 2N0σ2(2.4–28) Chernov Bound for Sums of Random Variables Let{Xi},1≤i≤n, denote a sequence of iid random variables and deﬁne Y=1 nn⎦summationdisplay i=1Xi (2.4–29) Proakis-27466 book September 25, 2007 13:9 62 Digital Communications We are interested to ﬁnd a bound on P [Y>δ], where δ>E[X]. Applying the Chernov bound, we have P[Y>δ]=P⎦bracketleftBiggn⎦summationdisplay i=1Xi>nδ⎦bracketrightBigg ≤E⎦bracketleftbigg eν⎦parenleftbig⎦summationtextn i=1Xi−nδ⎦parenrightbig⎦bracketrightbigg =⎦bracketleftbigE⎦bracketleftbigeν(X−δ)⎦bracketrightbig⎦bracketrightbign,ν > 0(2.4–30) To ﬁnd the optimal choice of νwe equate the derivative of the right-hand side to zero d dν⎦bracketleftbigE⎦bracketleftbigeν(X−δ)⎦bracketrightbig⎦bracketrightbign=n⎦bracketleftbigE⎦bracketleftbigeν(X−δ)⎦bracketrightbig⎦bracketrightbign−1E⎦bracketleftbig(X−δ)eν(X−δ)⎦bracketrightbig=0 (2.4–31) The single root of this equation is obtained by solving E⎦bracketleftbigXeνX⎦bracketrightbig=δE⎦bracketleftbigeνX⎦bracketrightbig(2.4–32) which is exactly Equation 2.4–7. Therefore, for the sum of iid random variables we ﬁnd the ν∗solution of Equation 2.4–7, and then we use P[Y>δ]≤⎦bracketleftBig E⎦bracketleftBig eν∗(X−δ)⎦bracketrightBig⎦bracketrightBign =e−nν∗δ⎦bracketleftBig E⎦bracketleftBig eν∗X⎦bracketrightBig⎦bracketrightBign (2.4–33) EXAMPLE 2.4–3. The Xi’s are binary iid random variables with P [X=1]=1− P[X=−1]=p, where p<1 2. We are interested to ﬁnd a bound on P⎦bracketleftBiggn⎦summationdisplay i=1Xi>0⎦bracketrightBigg We have E [X]=p−(1−p)=2p−1<0. Assuming δ=0, the condition δ>E[X] is satisﬁed, and the preceding development can be applied to this case. We have E⎦bracketleftbig XeνX⎦bracketrightbig =peν−(1−p)e−ν(2.4–34) and Equation 2.4–7 becomes peν−(1−p)e−ν=0 (2.4–35) which has the unique solution ν∗=1 2ln1−p p(2.4–36) Using this value, we have E⎦bracketleftbig eν∗X⎦bracketrightbig =p⎦radicalBigg 1−p p+(1−p)⎦radicalbiggp 1−p=2⎦radicalbig p(1−p) (2.4–37) Substituting this result into Equation 2.4–33 results in P⎦bracketleftBiggn⎦summationdisplay i=1Xi>0⎦bracketrightBigg ≤[4p(1−p)]n 2 (2.4–38) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 63 Since for p<1 2we have 4 p(1−p)<1, the bound given in Equation 2.4–38 tends to zero exponentially. 2.5 LIMIT THEOREMS FOR SUMS OF RANDOM V ARIABLES If{Xi,i=1,2,3,...} represents a sequence of iid random variables, then it is intu- itively clear that the running average of this sequence, i.e., Yn=1 nn⎦summationdisplay i=1Xi (2.5–1) should in some sense converge to the average of the random variables. Two limit theorems, i.e., the law of large numbers (LLN) and the central limit theorem (CLT), rigorously state how the running average of the random variable behaves as nbecomes large. The (strong) law of large numbers states that if {Xi,i=1,2,...} is a sequence of iid random variables with E [X1]<∞, then 1 nn⎦summationdisplay i=1Xi−→ E[X1] (2.5–2) where the type of convergence is convergence almost everywhere (a.e.) or convergence almost surely (a.s.), meaning the set of points in the probability space for which the left-hand side does not converge to the right-hand side has zero probability. The central limit theorem states that if {Xi,i=1,2,...} is a sequence of iid random variables with m=E[X1]<∞andσ2=VA R [X1]<∞, then we have 1 n⎦summationtextn i=1Xi−m σ√n−→N(0,1) (2.5–3) The type of convergence in the CLT is convergence in distribution, meaning the CDF of the left-hand side converges to the CDF of N(0,1) as nincreases. 2.6 COMPLEX RANDOM V ARIABLES A complex random variable Z=X+jYcan be considered as a pair of real random variables XandY. Therefore, we treat a complex random variable as a two-dimensional random vector with components XandY. The PDF of a complex random variable is deﬁned to be the joint PDF of its real and complex parts. If XandYare jointly Gaussian random variables, then Zis a complex Gaussian random variable. The PDF of a zero-mean complex Gaussian random variable Zwith iid real and imaginary parts Proakis-27466 book September 25, 2007 13:9 64 Digital Communications is given by p(z)=1 2πσ2e−x2+y2 2σ2 (2.6–1) =1 2πσ2e−|z|2 2σ2 (2.6–2) For a complex random variable Z, the mean and variance are deﬁned by E[Z]=E[X]+jE[Y] (2.6–3) VA R [Z]=E⎦bracketleftbig|Z|2⎦bracketrightbig−|E[Z]|2=VA R [X]+VA R [Y] (2.6–4) 2.6–1 Complex Random Vectors A complex random vector is deﬁned as Z=X+jY, where XandYare real-valued random vectors of size n. We deﬁne the following real-valued matrices for a complex random vector Z. CX=E⎦bracketleftbig(X−E[X])(X−E[X])t⎦bracketrightbig(2.6–5) CY=E⎦bracketleftbig(Y−E[Y])(Y−E[Y])t⎦bracketrightbig(2.6–6) CXY=E⎦bracketleftbig(X−E[X])(Y−E[Y])t⎦bracketrightbig(2.6–7) CYX=E⎦bracketleftbig(Y−E[Y])(X−E[X])t⎦bracketrightbig(2.6–8) Matrices CXandCYare the covariance matrices of real random vectors XandY, respectively, and hence they are symmetric and nonnegative deﬁnite. It is clear from above that CYX=Ct XY. The PDF of Zis the joint PDF of its real and imaginary parts. If we deﬁne the 2n-dimensional real vector ˜Z=⎦bracketleftBigg X Y⎦bracketrightBigg (2.6–9) then the PDF of the complex vector Zis the PDF of the real vector ˜Z. It is clear that C˜Z, the covariance matrix of ˜Z, can be written as C˜Z=⎦bracketleftBigg CX CXY CYX CY⎦bracketrightBigg (2.6–10) We also deﬁne the following two, in general complex-valued, matrices CZ=E⎦bracketleftbig(Z−E[Z])(Z−E[Z])H⎦bracketrightbig(2.6–11) ⎦tildewideCZ=E⎦bracketleftbig(Z−E[Z])(Z−E[Z])t⎦bracketrightbig(2.6–12) where Atdenotes the transpose and AHdenotes the Hermitian transpose of A(Ais transposed and each element of it is conjugated). CZand⎦tildewideCZare called the covariance and the pseudocovariance of the complex random vector Z, respectively. It is easy to Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 65 verify that for any Z, the covariance matrix is Hermitian†and nonnegative deﬁnite. The pseudocovariance is skew-Hermitian . From these deﬁnitions it is easy to verify the following relations. CZ=CX+CY+j(CYX−CXY) (2.6–13) ⎦tildewideCZ=CX−CY+j(CXY+CYX) (2.6–14) CX=1 2Re [CZ+⎦tildewideCZ] (2.6–15) CY=1 2Re [CZ−⎦tildewideCZ] (2.6–16) CYX=1 2Im [CZ+⎦tildewideCZ] (2.6–17) CXY=1 2Im [⎦tildewideCZ−CZ] (2.6–18) Proper and Circularly Symmetric Random Vectors A complex random vector Zis called proper if its pseudocovariance is zero, i.e., if ⎦tildewideCZ=0. From Equation 2.6–14 it is clear that for a proper random vector we have CX=CY (2.6–19) CXY=−CYX (2.6–20) Substituting these results into Equations 2.6–13 to 2.6–18 and 2.6–10, we conclude that for proper random vectors CZ=2CX+2jCYX (2.6–21) CX=CY=1 2Re[CZ] (2.6–22) CYX=−CXY=1 2Im[CZ] (2.6–23) C˜Z=⎦bracketleftBigg CX CXY −CXY CX⎦bracketrightBigg (2.6–24) For the special case of n=1, i.e., when we are dealing with a single complex random variable Z=X+jY, the conditions for being proper become VA R [X]=VA R [Y] (2.6–25) COV [X,Y]=−COV [Y,X] (2.6–26) which means that Zis proper if XandYhave equal variances and are uncorrelated. In this case V AR [Z]=2V A R [X]. Since in the case of jointly Gaussian random variables uncorrelated is equivalent to independent, we conclude that a complex Gaussian random †Matrix Ais Hermitian if A=AH. It is skew-Hermitian if AH=− A. Proakis-27466 book September 25, 2007 13:9 66 Digital Communications variable Zis proper if and only if its real and complex parts are independent with equal variance. For a zero-mean proper complex Gaussian random variable, the PDF is given by Equation 2.6–2. If the complex random vector Z=X+jYis Gaussian, meaning that XandY are jointly Gaussian, then we have p(z)=p(˜z)=1 (2π)n(detC˜Z)1 2e−1 2(˜z−˜m)tC˜Z−1(˜z−˜m)(2.6–27) where ˜m=E⎦bracketleftbig˜Z⎦bracketrightbig(2.6–28) It can be shown that in the special case where Zis a proper n-dimensional complex Gaussian random vector, with mean m=E[Z]and nonsingular covariance matrix CZ, its PDF can be written as p(z)=1 πndetCZe−1 2(z−m)†CZ−1(z−m)(2.6–29) A complex random vector Zis called circularly symmetric orcircular if rotating the vector by any angle does not change its PDF. In other words, a complex random vector Zis circularly symmetric if ZandejθZhave the same PDF for all θ. In Problem 2.34 we will see that if Zis circular, then it is zero-mean and proper, i.e., E [Z]=0and E⎦bracketleftbigZZt⎦bracketrightbig=0. In Problem 2.35 we show that if Zis a zero-mean proper Gaussian complex vector, then Zis circular. In other words, for complex Gaussian random vectors being zero-mean and proper is equivalent to being circular. In Problem 2.36 we show that if Zis a proper complex vector, then any afﬁne transformation of it, i.e., any transform of the form W=AZ+b, is also a proper complex vector. Since we know that if Zis Gaussian, so is W, we conclude that if Zis a proper Gaussian vector, so is W. For more details on properties of proper and circular random variables and random vectors, the reader is referred to Neeser and Massey(1993) and Eriksson and Koivunen (2006). 2.7 RANDOM PROCESSES Random processes, stochastic processes, or random signals are fundamental in the studyof communication systems. Modeling information sources and communication chan-nels requires a good understanding of random processes and techniques for analyzingthem. We assume that the reader has a knowledge of the basic concepts of randomprocesses including deﬁnitions of mean, autocorrelation, cross-correlation, stationar-ity, and ergodicity as given in standard texts such as Leon-Garcia (1994), Papoulis andPillai (2002), Stark and Woods (2002). In the following paragraphs we present a briefreview of the most important properties of random processes. Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 67 The mean mX(t) and the autocorrelation function of a random process X(t) are deﬁned as mX(t)=E[X(t)] (2.7–1) RX(t1,t2)=E⎦bracketleftbigX(t1)X∗(t2)⎦bracketrightbig(2.7–2) Thecross-correlation function of two random processes X(t) and Y(t) is deﬁned by RXY(t1,t2)=E⎦bracketleftbigX(t1)Y∗(t2)⎦bracketrightbig(2.7–3) Note that RX(t2,t1)=R∗ X(t1,t2), i.e., RX(t1,t2) is Hermitian. For the cross-correlation we have RYX(t2,t1)=R∗ XY(t1,t2). 2.7–1 Wide-Sense Stationary Random Processes Random process X(t)i swide-sense stationary (WSS) if its mean is constant and RX(t1,t2)=RX(τ) where τ=t1−t2. For WSS processes RX(−τ)=R∗ X(τ). Two processes X(t) and Y(t) are jointly wide-sense stationary if both X(t) and Y(t) are WSS and RXY(t1,t2)=RXY(τ). For jointly WSS processes RYX(−τ)=R∗ XY(τ). A complex process is WSS if its real and imaginary parts are jointly WSS. Thepower spectral density (PSD) or power spectrum of a WSS random process X(t) is a function SX(f) describing the distribution of power as a function of frequency. The unit for power spectral density is watts per hertz. The Wiener-Khinchin theorem states that for a WSS process, the power spectrum is the Fourier transform of the autocorrelation function RX(τ), i.e., SX(f)=F[RX(τ)] (2.7–4) Similarly, the cross spectral density (CSD) of two jointly WSS processes is deﬁned as the Fourier transform of their cross-correlation function. SXY(f)=F[RXY(τ)] (2.7–5) The cross spectral density satisﬁes the following symmetry property: SXY(f)=S∗ YX(f) (2.7–6) From properties of the autocorrelation function it is easy to verify that the power spectral density of any real WSS process X(t) is a real, nonnegative, and even function of f. For complex processes, power spectrum is real and nonnegative, but not necessarily even. The cross spectral density can be a complex function, even when both X(t) and Y(t) are real processes. IfX(t) and Y(t) are jointly WSS random processes, then Z(t)=aX(t)+bY(t)i s a WSS random process with autocorrelation and power spectral density given by RZ(τ)=|a|2RX(τ)+|b|2RY(τ)+ab∗RXY(τ)+ba∗RYX(τ) (2.7–7) SZ(f)=|a|2SX(f)+|b|2SY(f)+2R e[ ab∗SXY(f)] (2.7–8) Proakis-27466 book September 25, 2007 13:9 68 Digital Communications In the special case where a=b=1, we have Z(t)=X(t)+Y(t), which results in RZ(τ)=RX(τ)+RY(τ)+RXY(τ)+RYX(τ) (2.7–9) SZ(f)=SX(f)+SY(f)+2R e[SXY(f)] (2.7–10) and when a=1 and b=j,w eh a v e Z(t)=X(t)+jY(t) and RZ(τ)=RX(τ)+RY(τ)+j(RYX(τ)+RXY(τ)) (2.7–11) SZ(f)=SX(f)+SY(f)+2I m[SXY(f)] (2.7–12) When a WSS process X(t) passes through an LTI system with impulse response h(t) and transfer function H(f)=F[h(t)], the output process Y(t) and X(t) are jointly WSS and the following relations hold: mY=mX⎦integraldisplay∞ −∞h(t)dt (2.7–13) RXY(τ)=RX(τ)⋆h∗(−τ) (2.7–14) RY(τ)=RX(τ)⋆h(τ)⋆h∗(−τ) (2.7–15) mY=mXH(0) (2.7–16) SXY(f)=SX(f)H∗(f) (2.7–17) SY(f)=SX(f)|H(f)|2(2.7–18) The power in a WSS process X(t) is the sum of the powers at all frequencies, and therefore it is the integral of the power spectrum over all frequencies. We can write PX=E⎦bracketleftbig|X(t)|2⎦bracketrightbig=RX(0)=⎦integraldisplay∞ −∞SX(f)df (2.7–19) Gaussian Random Processes A real random process X(t) is Gaussian if for all positive integers nand for all (t1,t2,..., tn), the random vector ( X(t1),X(t2),..., X(tn))tis a Gaussian random vec- tor; i.e., random variables {X(ti)}n i=1are jointly Gaussian random variables. Similar to jointly Gaussian random variables, linear ﬁltering of Gaussian random processesresults in a Gaussian random process, even when the ﬁltering is time-varying. Two real random processes X(t) and Y(t) are jointly Gaussian if for all positive integers n,mand all ( t 1,t2,..., tn), and ( t/prime 1,t/prime 2,..., t/prime m), the random vector (X(t1),X(t2),..., X(tn),Y(t/prime 1),Y(t/prime 2),..., Y(t/prime m))t is a Gaussian vector. For two jointly Gaussian random processes X(t) and Y(t), being uncorrelated, i.e., having RXY(t+τ,t)=E[X(t+τ)]E[Y(t)]for all tandτ (2.7–20) is equivalent to being independent. A complex process Z(t)=X(t)+jY(t) is Gaussian if X(t) and Y(t) are jointly Gaussian processes. Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 69 White Processes A process is called a white process if its power spectral density is constant for all frequencies; this constant value is usually denoted byN0 2. SX(f)=N0 2(2.7–21) Using Equation 2.7–19, we see that the power in a white process is inﬁnite, indicating that white processes cannot exist as a physical process. Although white processes are notphysically realizable processes, they are very useful, closely modeling some importantphysical phenomenon including the thermal noise . Thermal noise is the noise generated in electric devices by thermal agitation of electrons. Thermal noise can be closely modeled by a random process N(t) having the following properties: 1.N(t) is a stationary process. 2.N(t) is a zero-mean process. 3.N(t) is a Gaussian process. 4.N(t) is a white process whose power spectral density is given by SN(f)=N0 2=kT 2(2.7–22) where Tis the ambient temperature in kelvins and kisBoltzmann’s constant , equal to 38×10−23J/K. Discrete-Time Random Processes Discrete-time random processes have similar properties to continuous time processes. In particular the PSD of a WSS discrete-time random process is deﬁned as the discrete-time Fourier transform of its autocorrelation function SX(f)=∞⎦summationdisplay m=−∞RX(m)e−j2πfm(2.7–23) and the autocorrelation function can be obtained as the inverse Fourier transform of the power spectral density as RX(m)=⎦integraldisplay1/2 −1/2SX(f)ej2πfmdf (2.7–24) The power in a discrete-time random process is given by P=E⎦bracketleftbig|X(n)|2⎦bracketrightbig=RX(0)=⎦integraldisplay1/2 −1/2SX(f)df (2.7–25) Proakis-27466 book September 25, 2007 13:9 70 Digital Communications 2.7–2 Cyclostationary Random Processes A random process X(t)i scyclostationary if its mean and autocorrelation function are periodic functions with the same period T0. For a cyclostationary process we have mX(t+T0)=mX(t) (2.7–26) RX(t1+T0,t2+T0)=RX(t1,t2) (2.7–27) Cyclostationary processes are encountered frequently in the study of communi- cation systems because many modulated processes can be modeled as cyclostationary processes. For a cyclostationary process, the average autocorrelation function is deﬁnedas the average of the autocorrelation function over one period RX(τ)=1 T0⎦integraldisplayT0 0RX(t+τ,t)dt (2.7–28) The (average) power spectral density for a cyclostationary process is deﬁned as theFourier transform of the average autocorrelation function, i.e., SX(f)=F⎦bracketleftbigRX(τ)⎦bracketrightbig(2.7–29) EXAMPLE 2.7–1. Let{an}denote a discrete-time WSS random process with mean ma(n)=E[an]=maand autocorrelation function Ra(m)=E⎦bracketleftbig an+ma∗ n⎦bracketrightbig . Deﬁne the random process X(t)=∞⎦summationdisplay n=−∞ang(t−nT) (2.7–30) for an arbitrary deterministic function g(t). We have mX(t)=E[X(t)]=ma∞⎦summationdisplay n=−∞g(t−nT) (2.7–31) This function is obviously periodic with period T. For the autocorrelation function we have RX(t+τ,t)=∞⎦summationdisplay n=−∞∞⎦summationdisplay m=−∞E⎦bracketleftbig ana∗ m⎦bracketrightbig g(t+τ−nT)g∗(t−mT) (2.7–32) =∞⎦summationdisplay n=−∞∞⎦summationdisplay m=−∞Ra(n−m)g(t+τ−nT)g∗(t−mT) (2.7–33) It can readily be veriﬁed that RX(t+τ+T,t+T)=RX(t+τ,t) (2.7–34) Equations 2.7–31 and 2.7–34 show that X(t) is a cyclostationary process. Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 71 2.7–3 Proper and Circular Random Processes For a complex random process Z(t)=X(t)+jY(t), we deﬁne the covariance and the pseudocovariance, similar to the case of complex random vectors, as CZ(t+τ,t)=E[Z(t+τ)Z∗(t)] (2.7–35) ⎦tildewideCZ(t+τ,t)=E[Z(t+τ)Z(t)] (2.7–36) It is easy to verify that similar to Equations 2.6–13 and 2.6–14, we have CZ(t+τ,t)=CX(t+τ,t)+CY(t+τ,t)+j(CYX(t+τ,t)−CXY(t+τ,t)) (2.7–37) ⎦tildewideCZ(t+τ,t)=CX(t+τ,t)−CY(t+τ,t)+j(CYX(t+τ,t)+CXY(t+τ,t)) (2.7–38) A complex random process Z(t)i sproper if its pseudocovariance is zero, i.e., ⎦tildewideCZ(t+τ,t)=0. For a proper random process we have CX(t+τ,t)=CY(t+τ,t) (2.7–39) CYX(t+τ,t)=−CXY(t+τ,t) (2.7–40) and CZ(t+τ,t)=2CX(t+τ,t)+j2CYX(t+τ,t) (2.7–41) IfZ(t) is a zero-mean process, then all covariances in Equations 2.7–35 to 2.7–41 are substituted with auto- or cross-correlations. When Z(t) is WSS, all auto- and cross-correlations are functions of τonly. A proper Gaussian random process is a random process for which, for all nand all (t 1,t2,...,t n), the complex random vector (Z(t1),Z(t2),..., Z(tn))tis a proper Gaussian vector. A complex random process Z(t) is circular if for all θ,Z(t) and ejθZ(t) have the same statistical properties. Similar to the case of complex vectors, it can be shown that if Z(t) is circular, then it is both proper and zero-mean. For the case of Gaussian processes, being proper and zero-mean is equivalent to being circular. Also similar tothe case of complex vectors, passing a circular Gaussian process through a linear (notnecessarily time-invariant) system results in a circular Gaussian process at the output. 2.7–4 Markov Chains Markov chains are discrete-time, discrete-valued random processes in which the currentvalue depends on the entire past values only through the most recent values. In a jth- order Markov chain, the current value depends on the past values only through the mostrecent jvalues, i.e., P[X n=xn⎦vextendsingle⎦vextendsingleXn−1=xn−1,Xn−2=xn−2,... ] =P[Xn=xn⎦vextendsingle⎦vextendsingleXn−1=xn−1,Xn−2=xn−2,..., Xn−j=xn−j] (2.7–42) Proakis-27466 book September 25, 2007 13:9 72 Digital Communications It is convenient to consider the set of the most recent jvalues as the state of the Markov chain. With this deﬁnition the current state of the Markov chain,i.e., S n=(Xn,Xn−1,..., Xn−j+1), depends only on the most recent state Sn−1= (Xn−1,Xn−2,..., Xn−j). That is, P[Sn=sn|Sn−1=sn−1,Sn−2=sn−2,... ]=P[Sn=sn|Sn−1=sn−1] (2.7–43) which represents a ﬁrst-order Markov chain in terms of the state variable Sn. Note that with this notation, Xnis a deterministic function of state Sn. We can generalize this notion to the case where the state evolves according to Equation 2.7–43 but the output—or the value of the random process X n—depends on state Snthrough a conditional probability mass function P[Xn=xn|Sn=sn] (2.7–44) With this background, we deﬁne a Markov chain†as a ﬁnite-state machine with state at time n, denoted by Sn, taking values in the set {1,2,..., S}such that Equation 2.7–43 holds and the value of the random process at time n, denoted by Xnand taking values in a discrete set, depends statistically on the state through the conditional PMFP[X n=xn|Sn=sn]. The internal development of the process depends on the set of states and the proba- bilistic law that governs the transitions between the states. If P [ Sn|Sn−1] is independent ofn(time), the Markov chain is called homogeneous . In this case the probability of transition from state ito state j,1≤i,j≤S, is independent of nand is denoted byPij Pij=P[Sn=j|Sn−1=i] (2.7–45) In a homogeneous Markov chain, we deﬁne the state transition matrix ,o rone- step transition matrix ,Pas a matrix with elements Pij. The element at row iand column jdenotes the probability of a direct transition from state ito state j.Pis a matrix with nonnegative elements, and the sum of each row of it is equal to 1. Then-step transition matrix gives the probabilities of moving from itojinnsteps. For discrete-time homogeneous Markov chains, the n-step transition matrix is equal to P n. All Markov chains studied here are assumed to be homogeneous. The row vector p(n)=[p1(n)p2(n)···,pS(n)], where pi(n) denotes the prob- ability of being in state iat time n,i st h e state probability vector of the Markov chain at time n. From this deﬁnition it is clear that p(n)=p(n−1)P (2.7–46) and p(n)=p(0)Pn(2.7–47) †Strictly speaking, this is the deﬁnition of a ﬁnite-state Markov chain (FSMC), which is the only class of Markov chains studied in this book. Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 73 If lim n→∞Pnexists and all its rows are equal, we denote each row of the limit by p, i.e., lim n→∞Pn=⎡ ⎢⎢⎢⎢⎣p p ... p⎤ ⎥⎥⎥⎥⎦ (2.7–48) In this case lim n→∞p(n)=lim n→∞p(0)Pn=p(0)⎡ ⎢⎢⎢⎢⎣p p ... p⎤ ⎥⎥⎥⎥⎦=p (2.7–49) This means that starting from anyinitial probability vector p(0), the Markov chain stabilizes at the state probability vector given by p, which is called the steady-state , equilibrium ,o rstationary state probability distribution of the Markov chain. Since after reaching the steady-state probability distribution these probabilities do not change, p can be obtained as the solution of the equation pP=p (2.7–50) that satisﬁes the conditions pi≥0 and⎦summationtext ipi=1 (i.e., it is a probability vector). If a Markov chain starts from state p, then it will always remain in this state, because pP= p. Some basic questions are the following: Does pP=palways have a solution that is a probability vector? If yes, under what conditions is this solution unique? Under whatconditions does lim n→∞Pnexist? If the limit exists, does the limit have equal rows? If it is possible to move from any state of a Markov chain to any other state in a ﬁnite number of steps, the Markov chain is called irreducible . The period of state iof a Markov chain is the greatest common divisor (GCD) of all nsuch that Pii(n)>0. State iisaperiodic if its period is equal to 1. A ﬁnite-state Markov chain is called ergodic if it is irreducible and all its states are aperiodic. It can be shown that in an ergodic Markov chain lim n→∞Pnalways exists and all rows of the limit are equal, i.e., Equation 2.7–48 holds. In this case a unique sta-tionary (steady-state) state probability distribution exists and starting from any initialstate probability vector, the Markov chain ends up in the steady-state state probabilityvector p. EXAMPLE 2.7–2. A Markov chain with four states is described by the ﬁnite-state dia- gram shown in Figure 2.7–1. For this Markov chain we have P=⎡ ⎢⎢⎢⎣1 21 301 6 1 201 20 01 403 4 5 601 60⎤ ⎥⎥⎥⎦(2.7–51) Proakis-27466 book September 25, 2007 13:9 74 Digital Communications 41 32P12 /H11005 1 3 P11 /H11005 12 P 21 /H11005 12 P 43 /H11005 16 P 34 /H11005 34P 14 /H11005 16P 41 /H11005 56P 32 /H11005 14P 23 /H11005 12 FIGURE 2.7–1 State transition diagram for a FSMC. It is easily veriﬁed that this Markov chain is irreducible and aperiodic, and thus ergodic. To ﬁnd the steady-state probability distribution, we can either ﬁnd the limit of Pnas n→∞ or solve Equation 2.7–50. The result is p≈[0.49541 0.19725 0.12844 0.17889] (2.7–52) 2.8 SERIES EXPANSION OF RANDOM PROCESSES Series expansion of random processes results in expressing the random processes in terms of a sequence of random variables as coefﬁcients of orthogonal or orthonormalbasis functions. This type of expansion reduces working with random processes to work-ing with random variables, which in many cases are easier to handle. In the followingwe describe two types of series expansions for random processes. First we describe thesampling theorem for band-limited random processes, and then we continue with theKarhunen-Loeve expansion of random processes, which is a more general expansion. 2.8–1 Sampling Theorem for Band-Limited Random Processes A deterministic real signal x(t) with Fourier transform X(f) is called band-limited if X(f)=0 for|f|>W, where Wis the highest frequency contained in x(t). Such a signal is uniquely represented by samples of x(t) taken at a rate of fs≥2Wsamples/s. The minimum rate fN=2Wsamples/s is called the Nyquist rate. For complex- valued signals Wis one-half of the frequency support of the signal; i.e., if W1andW2 Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 75 are the lowest and the highest frequency components of the signal, respectively, then 2W=W2−W1. The signal can be perfectly reconstructed from its sampled values if the sampling rate is at least equal to 2 W. The difference, however, is that the sampled values are complex in this case, and for specifying each sample, two real numbers arerequired. This means that a real signal can be perfectly described in terms of 2 Wreal numbers per second, or it has 2 W degrees of freedom orreal dimensions per second. For a complex signal the number of degrees of freedom is 4 Wper second, which is equivalent to 2 W complex dimensions or 4Wreal dimensions per second. Sampling below the Nyquist rate results in frequency aliasing. The band-limited signal sampled at the Nyquist rate can be reconstructed from its samples by use of theinterpolation formula x(t)= ∞⎦summationdisplay n=−∞x⎦parenleftbiggn 2W⎦parenrightbigg sinc⎦bracketleftbigg 2W⎦parenleftbigg t−n 2W⎦parenrightbigg⎦bracketrightbigg (2.8–1) where {x(n/2W)}are the samples of x(t) taken at t=n/2W,n=0,±1,±2,.... Equivalently, x(t) can be reconstructed by passing the sampled signal through an ideal lowpass ﬁlter with impulse response h(t)=sinc(2 Wt). Figure 2.8–1 illustrates the signal reconstruction process based on ideal interpolation. Note that the expansion of x(t) as given by Equation 2.8–1 is an orthogonal expansion and not an orthonormal expansion since ⎦integraldisplay∞ −∞sinc⎦bracketleftbigg 2W⎦parenleftbigg t−n 2W⎦parenrightbigg⎦bracketrightbigg sinc⎦bracketleftbigg 2W⎦parenleftbigg t−m 2W⎦parenrightbigg⎦bracketrightbigg dt=⎦braceleftBigg 1 2Wn=m 0 n/negationslash=m(2.8–2) A stationary stochastic process X(t) is said to be band-limited if its power spec- tral density SX(f)=0 for|f|>W. Since SX(f) is the Fourier transform of the autocorrelation function RX(τ), it follows that RX(τ) can be represented as RX(τ)=∞⎦summationdisplay n=−∞RX⎦parenleftbiggn 2W⎦parenrightbigg sinc⎦bracketleftbigg 2W⎦parenleftbigg τ−n 2W⎦parenrightbigg⎦bracketrightbigg (2.8–3) where {RX(n/2W)}are samples of RX(τ) taken at τ=n/2W,n=0,±1,±2,.... Now, if X(t) is a band-limited stationary stochastic process, then X(t) can be repre- sented as X(t)=∞⎦summationdisplay n=−∞X⎦parenleftbiggn 2W⎦parenrightbigg sinc⎦bracketleftbigg 2W⎦parenleftbigg t−n 2W⎦parenrightbigg⎦bracketrightbigg (2.8–4) x(t) Sample of x(t) FIGURE 2.8–1 Sampling and reconstruction fromsamples. Proakis-27466 book September 25, 2007 13:9 76 Digital Communications where {X(n/2W)}are samples of X(t) taken at t=n/2W,n=0,±1,±2,.... This is the sampling representation for a stationary stochastic process. The samples are random variables that are described statistically by appropriate joint probability density func-tions. If X(t) is a WSS process, then random variables {X(n/2W)}represent a WSS discrete-time random process. The autocorrelation of the sample random variables isgiven by E⎦bracketleftbigg X⎦parenleftbiggn 2W⎦parenrightbigg X∗⎦parenleftbiggm 2W⎦parenrightbigg⎦bracketrightbigg =RX⎦parenleftbiggn−m 2W⎦parenrightbigg =⎦integraldisplayW −WSX(f)ej2πfn−m 2Wdf(2.8–5) If the process X(t) is ﬁltered white Gaussian noise, then it is zero-mean and its power spectrum is ﬂat in the [− W,W] interval. In this case the samples are uncorrelated, and since they are Gaussian, they are independent as well. The signal representation in Equation 2.8–4 is easily established by showing that (Problem 2.44) E⎡ ⎣⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingleX(t)−∞⎦summationdisplay n=−∞X⎦parenleftbiggn 2W⎦parenrightbigg sinc⎦bracketleftbigg 2W⎦parenleftbigg t−n 2W⎦parenrightbigg⎦bracketrightbigg⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle⎦vextendsingle2⎤ ⎦=0 (2.8–6) Hence, equality between the sampling representation and the stochastic process X(t) holds in the sense that the mean square error is zero. 2.8–2 The Karhunen-Lo` eve Expansion The sampling theorem presented above gives a straightforward method for orthogonal expansion of band-limited processes. In this section we present the Karhunen-Lo` eveexpansion, an orthonormal expansion that applies to a large class of random processesand results in uncorrelated random variables as expansion coefﬁcients. We present onlythe results of the Karhunen-Lo` eve expansion. The reader is referred to Van Trees (1968)or Lo` eve (1955) for details. There are many ways in which a random process can be expanded in terms of a sequence of random variables {X n}and an orthonormal basis {φn(t)}.H o w e v e r ,i fw e require the additional condition that the random variables Xnbe mutually uncorrelated, then the orthonormal bases have to be the solutions of an eigenfunction problem givenby an integral equation whose kernel is the autocovariance function of the randomprocess. Solving this integral equation results in the orthonormal basis {φ n(t)}, and projecting the random process on this basis results in the sequence of uncorrelatedrandom variables {X n}. The Karhunen-Lo` eve expansion states that under mild conditions, a random process X(t) with autocovariance function CX(t1,t2)=RX(t1,t2)−mX(t1)m∗ X(t2) (2.8–7) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 77 can be expanded over an interval of interest [ a,b] in terms of an orthonormal basis {φn(t)}∞ n=1such that the coefﬁcients of expansion are uncorrelated. The φn(t)’s are solutions (eigenfunctions) of the integral equation ⎦integraldisplayb aCX(t1,t2)φn(t2)dt2=λnφn(t1), a<t1<b (2.8–8) with appropriate normalization such that ⎦integraldisplayb a|φn(t)|2dt=1 (2.8–9) The Karhunen-Lo` eve expansion is given by ˆX(t)=∞⎦summationdisplay n=1Xnφn(t), a<t<b with the following properties: 1. Random variables Xndenoting the coefﬁcients of the expansion are projections of the random process X(t) on the basis functions, i.e., Xn=/angbracketleftX(t),φn(t)/angbracketright=⎦integraldisplayb aX(t)φ∗ n(t)dt (2.8–10) 2. Random variables Xnare mutually uncorrelated. Moreover, the variance of Xn isλn. COV [Xn,Xm]=⎦braceleftBigg λn n=m 0 n/negationslash=m(2.8–11) 3. We have E[ˆX(t)]=E[X(t)]=mX(t), a<t<b (2.8–12) 4.ˆX(t) is equal to X(t) in the mean square sense E[|X(t)−ˆX(t)|2]=0, a<t<b (2.8–13) 5. The covariance CX(t1,t2) can be expanded in terms of the bases and the eigenvalues as given in Equation 2.8–14. This is result is known as Mercer’s theorem . CX(t1,t2)=∞⎦summationdisplay n=1λnφn(t1)φn(t2), a<t1,t2<b (2.8–14) 6. The eigenfunctions {φn(t)}∞ n=1form a complete basis for expansion of all signals g(t) which have ﬁnite energy in the interval [ a,b]. In other words, if g(t) is such that ⎦integraldisplayb a|g(t)|2dt<∞ Proakis-27466 book September 25, 2007 13:9 78 Digital Communications then we can expand it in terms of {φn(t)}as g(t)=∞⎦summationdisplay n=1gnφn(t), a<t<b (2.8–15) where gn=/angbracketleftg(t),φ n(t)/angbracketright=⎦integraldisplayb ag(t)φ∗ n(t)dt (2.8–16) Equation 2.8–13, which states the Karhunen-Lo` eve expansion, is usually written in the form X(t)=∞⎦summationdisplay n=1Xnφn(t), a<t<b (2.8–17) where it is understood that the equality is in the mean square sense. The {φn(t)}are obtained by solving Equation 2.8–8 and normalizing the solutions, and the coefﬁcients {Xn}are obtained by using Equation 2.8–10. It is worthwhile noting that the Karhunen-Lo` eve expansion applies to both WSS and nonstationary processes. In the special case where the process is zero-mean, the autoco-variance function C X(t1,t2) is substituted with the autocorrelation function RX(t1,t2). If the process X(t) is a Gaussian process, {Xn}are independent Gaussian random variables. EXAMPLE 2.8–1. LetX(t) be a zero-mean white process with power spectral density N0 2. To derive the Karhunen-Lo` eve expansion for this process over an arbitrary interval [a,b], we have to solve the integral equation ⎦integraldisplayb aN0 2δ(t1−t2)φn(t2)dt2=λnφn(t1), a<t1<b (2.8–18) whereN0 2δ(t1−t2) is the autocorrelation function of the white process. Using the sifting property of the impulse function, we have N0 2φn(t1)=λnφn(t1), a<t1<b (2.8–19) From this equation we see that φn(t) can be any arbitrary function. Therefore, any orthonormal basis can be used for expansion of white processes, and all coefﬁcients of the expansion Xnwill have the same variance ofN0 2. 2.9 BANDPASS AND LOWPASS RANDOM PROCESSES In general, bandpass and lowpass random processes can be deﬁned as WSS processes X(t) for which the autocorrelation function RX(τ) is either a bandpass or a lowpass signal. Recall that the autocorrelation function is an ordinary deterministic function with a Fourier transform which represents the power spectral density of the random Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 79 process X(t). Therefore, for a bandpass process the power spectral density is located around frequencies ±f0, and for lowpass processes the power spectral density is located around zero frequency. To be more speciﬁc, we deﬁne a bandpass (or narrowband) process as a real, zero- mean, and WSS random process whose autocorrelation function is a bandpass signal. Inspired by Equations 2.1–11, we deﬁne the in-phase andquadrature components of a bandpass random process X(t)a s Xi(t)=X(t) cos 2 πf0t+⎦hatwideX(t) sin 2πf0t Xq(t)=⎦hatwideX(t) cos 2 πf0t−X(t) sin 2πf0t(2.9–1) We will now show that 1.Xi(t) and Xq(t) are jointly WSS zero-mean random processes. 2.Xi(t) and Xq(t) have the same power spectral density. 3.Xi(t) and Xq(t) are both lowpass processes; i.e., their power spectral density is located around f=0. We also deﬁne the lowpass equivalent process X l(t)a s Xl(t)=Xi(t)+jXq(t) (2.9–2) and we will derive an expression for its autocorrelation function and power spectral density. In addition we will see that Xl(t) is a proper random process. Since X(t) by assumption is zero-mean, so is ⎦hatwideX(t), its Hilbert transform. This is obvious since the Hilbert transform is just a ﬁltering operation. From this observation,it is clear that X i(t) and Xq(t) are both zero-mean processes. To derive the autocorrelation function of Xi(t), we have RXi(t+τ,t)=E[Xi(t+τ)Xi(t)] =E[ (X(t+τ) cos 2 πf0(t+τ)+⎦hatwideX(t+τ) sin 2πf0(t+τ)) ×(X(t) cos 2 πf0t+⎦hatwideX(t) sin 2πf0t)](2.9–3) Expanding this relation, we have RXi(t+τ,t)=RX(τ) cos 2 πf0(t+τ) cos 2 πf0t +RXˆX(t+τ,t) cos 2 πf0(t+τ) sin 2πf0t +RˆXX(t+τ,t) sin 2πf0(t+τ) cos 2 πf0t +RˆXˆX(t+τ,t) sin 2πf0(t+τ) sin 2πf0t(2.9–4) Since the Hilbert transform is the result of passing the process through an LTI sys- tem, we conclude that X(t) and ⎦hatwideX(t) are jointly WSS and therefore all the auto- and cross-correlations in Equation 2.9–4 are functions of τonly. Using Equations 2.7–17 Proakis-27466 book September 25, 2007 13:9 80 Digital Communications and 2.7–18, we can easily show that (see Problem 2.56) RXˆX(τ)=−⎦hatwideRX(τ) RˆXX(τ)=⎦hatwideRX(τ) RˆXˆX(τ)=RX(τ)(2.9–5) Substituting these results into Equation 2.9–4 and using standard trigonometric identities yield RXi(τ)=RX(τ) cos(2 πf0τ)+⎦hatwideRX(τ) sin(2 πf0τ) (2.9–6) Similarly, we can show that RXq(τ)=RXi(τ)=RX(τ) cos(2 πf0τ)+⎦hatwideRX(τ) sin(2 πf0τ) (2.9–7) RXiXq(τ)=− RXqXi(τ)=RX(τ) sin(2 πf0τ)−⎦hatwideRX(τ) cos(2 πf0τ) (2.9–8) These relations show that Xi(t) and Xq(t) are zero-mean jointly WSS processes with equal autocorrelation functions (and thus equal power spectral densities). To derive the common power spectral density of Xi(t) and Xq(t) and their cross spectral density, we derive the Fourier transforms of Equations 2.9–7 and 2.9–8. Weneed to use the modulation property of the Fourier transform and the fact that the Fourier transform of ⎦hatwideRX(τ) is equal to −jsgn( f)SX(f). Given these facts, it is straightforward to derive SXi(f)=SXq(f)=⎦braceleftBigg SX(f+f0)+SX(f−f0) |f|<f0 0 otherwise(2.9–9) SXiXq(f)=−SXqXi(f)=⎦braceleftBigg j[SX(f+f0)−SX(f−f0)] |f|<f0 0 otherwise (2.9–10) Equation 2.9–9 states that the common power spectral density of the in-phase and quadrature components of X(t) is obtained by shifting the power spectral density of X(t) to left and right by f0and adding the results and then removing all components outside [−f0,f0]. This result also shows that both Xi(t) and Xq(t) are lowpass processes. From Equation 2.9–10 we see that if SX(f+f0)=SX(f−f0) for|f|<f0, then SXiXq(f)=0 and consequently, RXiXq(τ)=0. Since Xi(t) and Xq(t) are zero-mean processes, from RXiXq(τ)=0 we conclude that under this condition Xi(t) and Xq(t) are uncorrelated. One of the cases where we have SX(f+f0)=SX(f−f0) for|f|<0 occurs when SX(f) is symmetric around f0, in which case the in-phase and quadrature components will be uncorrelated processes. We deﬁne the complex process Xl(t)=Xi(t)+jXq(t) as the lowpass equivalent ofX(t). Since Xi(t) and Xq(t) are both lowpass processes, we conclude that Xl(t)i s also a lowpass process. Comparing Equations 2.9–7 and 2.9–8 with Equations 2.7–39and 2.7–40, we can conclude that X l(t) is a proper random process, and therefore, from Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 81 Equation 2.7–41, we have RXl(τ)=2RXi(τ)+2jRXqXi(τ) (2.9–11) =2[RX(τ)+j⎦hatwideRX(τ)]e−j2πf0t(2.9–12) where we have used Equations 2.9–7 and 2.9–8. Comparing Equations 2.9–12 and 2.1–6, we observe that RXl(τ) is twice the lowpass equivalent of RX(τ). In other words, the autocorrelation function of the lowpass equivalent process X l(t)is twice the lowpass equivalent of the autocorrelation function of the bandpass process X (t). Taking Fourier transform of both sides of Equation 2.9–12, we obtain SXl(f)=⎦braceleftBigg 4SX(f+f0)|f|<f0 0 otherwise(2.9–13) and consequently, SX(f)=1 4[SXl(f−f0)+SXl(−f−f0)] (2.9–14) We also observe that if X(t) is a Gaussian process, then Xi(t),Xq(t), and Xl(t) will be jointly Gaussian processes; and since Xl(t) is Gaussian, zero-mean, and proper, we conclude that Xl(t) is a circular process as well. In this case if SX(f+f0)=SX(f−f0) for|f|<f0, then Xi(t) and Xq(t) will be independent processes. EXAMPLE 2.9–1. White Gaussian noise with power spectral density ofN0 2passes through an ideal bandpass ﬁlter with transfer function H(f)=⎦braceleftbigg1 |f−f0|<W 0 otherwise where W<f0. The output, called ﬁltered white noise , is denoted by X(t). This process has a power spectral density of SX(f)=⎦braceleftbiggN0 2|f−f0|<W 0 otherwise Since SX(f+f0)=SX(f−f0) for|f|<f0, and the process is Gaussian, Xi(f) and Xq(f) are independent lowpass processes. Using Equation 2.9–9, we conclude that SXi(f)=SXq(f)=⎦braceleftbiggN0|f|<W 0 otherwise and from Equation 2.9–13, we conclude that SXl(f)=⎦braceleftbigg2N0|f|<W 0 otherwise Proakis-27466 book September 25, 2007 13:9 82 Digital Communications 2.10 BIBLIOGRAPHICAL NOTES AND REFERENCES In this chapter we have provided a review of basic concepts and deﬁnitions in signal analysis, the theory of probability, and stochastic processes. An advanced book on signalanalysis that covers most of the material presented here in detail is the book by Franks(1969). The texts by Davenport and Root (1958), Davenport (1970), Papoulis and Pillai(2002), Peebles (1987), Helstrom (1991), Stark and Woods (2002), and Leon-Garcia(1994) provide engineering-oriented treatments of probability and stochastic processes.A more mathematical treatment of probability theory may be found in the text by Lo` eve(1955). Finally, we cite the book by Miller (1964), which treats multidimensionalGaussian distributions. PROBLEMS 2.1Prove the following properties of Hilbert transforms: a.I f x(t)=x(−t), then ˆx(t)=− ˆx(−t). b.I f x(t)=−x(−t), then ˆx(t)=ˆx(−t). c.I f x(t)=cosω0t, then ˆx(t)=sinω0t. d.I f x(t)=sinω0t, then ˆx(t)=− cosω0t. e.ˆˆx(t)=−x(t) f.⎦integraldisplay∞ −∞x2(t)dt=⎦integraldisplay∞ −∞ˆx2(t)dt g.⎦integraldisplay∞ −∞x(t)ˆx(t)dt=0 2.2Letx(t) and y(t) denote two bandpass signals, and let xl(t) and yl(t) denote their lowpass equivalents with respect to some frequency f0. We know that in general xl(t) and yl(t) are complex signals. 1. Show that⎦integraldisplay∞ −∞x(t)y(t)dt=1 2Re⎦bracketleftbigg⎦integraldisplay∞ −∞xl(t)y∗ l(t)dt⎦bracketrightbigg 2. From this conclude that Ex=1 2Exl, i.e., the energy in a bandpass signal is one-half the energy in its lowpass equivalent. 2.3Suppose that s(t) is either a real- or complex-valued signal that is represented as a linear combination of orthonormal functions {fn(t)}, i.e., ˆs(t)=K⎦summationdisplay k=1skfk(t) where⎦integraldisplay∞ −∞fn(t)f∗ m(t)dt=⎦braceleftbigg 1 m=n 0 m/negationslash=n Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 83 Determine the expressions for the coefﬁcients {sk}in the expansion ˆsi(t) that minimize the energy Ee=⎦integraldisplay∞ −∞|s(t)−ˆs(t)|2dt and the corresponding residual error Ee. 2.4Suppose that a set of Msignal waveforms {slm(t)}is complex-valued. Derive the equations for the Gram-Schmidt procedure that will result in a set of N≤Morthonormal signal waveforms. 2.5Carry out the Gram-Schmidt orthogonalization of the signals in Figure 2.2–1(a) in the order s4(t),s3(t),s1(t), and thus obtain a set of orthonormal functions {fm(t)}. Then determine the vector representation of the signals {sn(t)}by using the orthonormal functions {fm(t)}. Also determine the signal energies. 2.6Assuming that the set of signals {φnl(t),n=1,..., N}is an orthonormal basis for rep- resentation of {sml(t),m=1,..., M}, show that the set of functions given by Equa- tion 2.2–54 constitutes a 2 Northonormal basis that is sufﬁcient for representation of M bandpass signals given in Equation 2.2–55. 2.7Show that ⎦tildewideφ(t)=−⎦hatwideφ(t) where ⎦hatwideφ(t) denotes the Hilbert transform and φand⎦tildewideφare given by Equation 2.2–54. 2.8Determine the correlation coefﬁcients ρkmamong the four signal waveforms {si(t)}shown in Figure 2.2–1 and their corresponding Euclidean distances. 2.9Prove that sl(t) is generally a complex-valued signal, and give the condition under which it is real. Assume that s(t) is a real-valued bandpass signal. 2.10 Consider the three waveforms fn(t) shown in Figure P2.10. FIGURE P2.10 Proakis-27466 book September 25, 2007 13:9 84 Digital Communications a. Show that these waveforms are orthonormal. b. Express the waveform x(t) as a linear combination of fn(t),n=1,2,3, if x(t)=⎧ ⎨ ⎩−10 ≤t<1 11 ≤t<3 −13 ≤t<4 and determine the weighting coefﬁcients. 2.11 Consider the four waveforms shown in Figure P2.11. a. Determine the dimensionality of the waveforms and a set of basis functions. b. Use the basis functions to represent the four waveforms by vectors s1,s2,s3, and s4. c. Determine the minimum distance between any pair of vectors. FIGURE P2.11 2.12 Determine a set of orthonormal functions for the four signals shown in Figure P2.12. FIGURE P2.12 2.13 A random experiment consists of drawing a ball from an urn that contains 4 red balls numbered 1, 2, 3, 4 and three black balls numbered 1, 2, 3. The following events aredeﬁned. 1.E 1=The number on the ball is even. 2.E2=The color of the ball is red, and its number is greater than 1. 3.E3=The number on the ball is less than 3. 4.E4=E1∪E3 5.E5=E1∪(E2∩E3) Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 85 Answer the following questions. 1. What is P(E2)? 2. What is P(E3|E2)? 3. What is P(E2|E4E3)? 4. Are E3andE5independent? 2.14 In a certain city three car brands A, B, C have 20%, 30% and 50% of the market share, respectively. The probability that a car needs major repair during its ﬁrst year of purchasefor the three brands is 5%, 10%, and 15%, respectively.1. What is the probability that a car in this city needs major repair during its ﬁrst year of purchase? 2. If a car in this city needs major repair during its ﬁrst year of purchase, what is the probability that it is made by manufacturer A? 2.15 The random variables X i,i=1,2,. . . , n, have joint PDF p(x1,x2,..., xn). Prove that p(x1,x2,x3,..., xn)=p(xn|xn−1,..., x1)p(xn−1|xn−2,..., x1)···p(x3|x2,x1)p(x2|x1)p(x1) 2.16 A communication channel with binary input and ternary output alphabets is shown in Figure P2.16. The probability of the input being 0 is 0.4. The transition probabilities areshown on the ﬁgure. A 0 11/H114084 1/H114083 1/H1140831/H1140831/H1140821/H114084 B CFIGURE P2.16 1. If the channel output is A, what is the best decision on channel input that minimizes the error probability? Repeat for the cases where channel output is B and C. 2 . I fa0i s transmitted and an optimal decision scheme (the one derived in part 1) is used at the receiver, what is the probability of error? 3. What is the overall error probability for this channel if the optimal decision scheme is used at the receiver. 2.17 The PDF of a random variable Xisp(x). A random variable Yis deﬁned as Y=aX+b where a<0. Determine the PDF of Yin terms of the PDF of X. 2.18 Suppose that Xis a Gaussian random variable with zero mean and unit variance. Let Y=aX3+b, a>0 Determine and plot the PDF of Y . Proakis-27466 book September 25, 2007 13:9 86 Digital Communications 2.19 The noise voltage in an electric circuit can be modeled as a Gaussian random variable with mean equal to zero and variance equal to 10−8. 1. What is the probability that the value of the noise exceeds 10−4? What is the probability that it exceeds 4 ×10−4? What is the probability that the noise value is between −2× 10−4and 10−4? 2. Given that the value of the noise is positive, what is the probability that it exceeds 10−4? 2.20 Xis aN(0,σ2) random variable. This random variable is passed through a system whose input-output relation is given by y=g(x). Find the PDF or the PMF of the output random variable Yin each of the following cases. 1. Square-law device, g(x)=ax2. 2. Limiter, g(x)=⎧ ⎨ ⎩−bx ≤−b bx ≥b x |x|<b 3. Hard limiter, g(x)=⎧ ⎨ ⎩ax >0 0 x=0 bx <0 4. Quantizer, g(x)=xnforan≤x<an+1,1≤n≤N, where xnlies in the interval [an,an+1] and the sequence {a1,a2,..., aN+1}satisﬁes the conditions a1=− ∞ , aN+1=∞ and for i>jwe have ai>aj. 2.21 Shows that for an N(m,σ2) random variable we have E[ (X−m)n]=⎦braceleftBigg 1×3×5×···× (2k−1)σ2k=(2k)!σ2k 2kk!forn=2k 0 for n=2k+1 2.22 a. Let XrandXibe statistically independent zero-mean Gaussian random variables with identical variance. Show that a (rotational) transformation of the form Yr+jYi=(Xr+jXi)ejφ results in another pair ( Yr,Yi) of Gaussian random variables that have the same joint PDF as the pair ( Xr,Xi). b. Note that ⎦bracketleftbiggYr Yi⎦bracketrightbigg =A⎦bracketleftbiggXr Xi⎦bracketrightbigg where Ai sa2×2 matrix. As a generalization of the two-dimensional transformation of the Gaussian random variables considered in ( a), what property must the linear transformation Asatisfy if the PDFs for XandY, where Y=AX,X=(X1X2···Xn), andY=(Y1Y2···Yn) are identical? 2.23 Show that if Xis a Gaussian vector, the random vector Y=AX, where the invertible matrix Arepresents a linear transformation, is also a Gaussian vector whose mean and Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 87 covariance matrix are given by mY=Am X CY=AC XAt 2.24 The random variable Yis deﬁned as Y=n⎦summationdisplay i=1Xi where the Xi,i=1,2,. . . , n, are statistically independent random variables with Xi=⎦braceleftbigg 1 with probability p 0 with probability 1 −p a. Determine the characteristic function of Y. b. From the characteristic function, determine the moments E(Y) and E(Y2). 2.25 This problem provides some useful bounds on Q(x). 1. By integrating e−u2+v2 2on the region u>xandv> xinR2, where x>0, then changing to polar coordinates and upper bounding the integration region by the region r>√ 2xin the ﬁrst quadrant, show that Q(x)≤1 2e−x2 2for all x≥0. 2. Apply integration by parts to ⎦integraldisplay∞ xe−y2 2dy y2 and show that x√ 2π(1+x2)e−x2 2<Q(x)<1√ 2πxe−x2 2 for all x>0. 3. Based on the result of part 2 show that, for large x, Q(x)≈1 x√ 2πe−x2 2 2.26 LetX1,X2,X3,... denote iid random variables each uniformly distributed on [0 ,A], where A>0. Let Yn=min{X1,X2,..., Xn}. 1. What is the PDF of Yn? 2. Show that if both Aandngo to inﬁnity such thatn A=λ, where λ> 0 is a constant, the density function of Yntends to an exponential density function. Specify this density function. 2.27 The four random variables X1,X2,X3,X4are zero-mean jointly Gaussian random variables with covariance Cij=E(XiXj) and characteristic function /Phi1X(ω1,ω2,ω3,ω4). Show that E(X1X2X3X4)=C12C34+C13C24+C14C23 Proakis-27466 book September 25, 2007 13:9 88 Digital Communications 2.28 Let /Theta1X(t)=E⎦bracketleftbig etX⎦bracketrightbig denote the moment generating function of random variable X. 1. Using the Chernov bound, show that ln P[X≥α]≤− max t≥0(αt−ln/Theta1X(t)) 2. Deﬁne I(α)=max t≥0(αt−ln/Theta1X(t)) as the large-deviation rate function of the random variable X, and let X1,X2,..., Xn be iid. Deﬁne Sn=(X1+X2+···+ Xn)/n. Show that for α≥E[X] 1 nln P[Sn≥α]≤− I(α) or equivalently P[Sn≥α]≤e−nI(α) Note: It can be shown that for α≥E[X],w eh a v e P[Sn≥α]=e−nI(α)+o(n), where o(n)→0a sn→∞ . This result is known as the large-deviation theorem . 3. Now assume the Xi’s are exponential, i.e., pX(x)=⎦braceleftbigg e−xx≥0 0 otherwise Using the large-deviation result, show that P[Sn≥α]=αne−n(α−1)+o(n) forα≥1. 2.29 From the characteristic functions for the central chi-square and noncentral chi-square random variables given in Table 2.3–3, determine their corresponding ﬁrst and secondmoments. 2.30 The PDF of a Cauchy distributed random variable Xis p(x)=a/π x2+a2,−∞<x<∞ a. Determine the mean and variance of X. b. Determine the characteristic function of X. 2.31 LetR0denote a Rayleigh random variable with PDF fR0(r0)=⎦braceleftBigg r0 σ2e−r2 0 2σ2 r0≥0 0 otherwise Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 89 andR1be Ricean with PDF fR1(r1)=⎦braceleftBigg r1 σ2I0⎦parenleftbigμr1 σ2⎦parenrightbig e−r2 1+μ2 2σ2 r1≥0 0 otherwise Furthermore, assume that R0andR1areindependent . Show that P(R0>R1)=1 2e−μ2 4σ2 2.32 Suppose that we have a complex-valued Gaussian random variable Z=X+jY, where (X,Y) are statistically independent variables with zero mean and variance E⎦bracketleftbig X2⎦bracketrightbig = E⎦bracketleftbig Y2⎦bracketrightbig =σ2. Let R=Z+m, where m=mr+jmiand deﬁne RasR=A+jB. Clearly, A=X+mrand B=Y+mi. Determine the following probability density functions:1.p A,B(a,b) 2.pU,/Phi1(u,φ), where U=√ A2+B2and/Phi1=tan−1B/A 3.pU(u) Note : In part 2 it is convenient to deﬁne θ=tan−1(mi/mr) so that mr=⎦radicalBig m2r+m2 icosθ, mi=⎦radicalBig m2r+m2 isinθ Furthermore, you must use Equation 2.3–34, deﬁning I0(·) as the modiﬁed Bessel function of order zero. 2.33 The random variable Yis deﬁned as Y=1 nn⎦summationdisplay i=1Xi where Xi,i=1,2,..., n, are statistically independent and identically distributed random variables each of which has the Cauchy PDF given in Problem 2.30.a. Determine the characteristic function of Y. b. Determine the PDF of Y. c. Consider the PDF of Yin the limit as n→∞ . Does the central limit theorem hold? Explain your answer. 2.34 Show that if Zis circular, then it is zero-mean and proper, i.e., E [Z]=0and E⎦bracketleftbig ZZt⎦bracketrightbig =0. 2.35 Show that if Zis a zero-mean proper Gaussian complex vector, then Zis circular. 2.36 Show that if Zis a proper complex vector, then any transform of the form W=AZ+b is also a proper complex vector. 2.37 Assume that random processes X(t) and Y(t) are individually and jointly stationary. a. Determine the autocorrelation function of Z(t)=X(t)+Y(t). b. Determine the autocorrelation function of Z(t) when X(t) and Y(t) are uncorrelated. c. Determine the autocorrelation function of Z(t) when X(t) and Y(t) are uncorrelated and have zero means. Proakis-27466 book September 25, 2007 13:9 90 Digital Communications 2.38 The autocorrelation function of a stochastic process X(t)i s RX(τ)=1 2N0δ(τ) Such a process is called white noise . Suppose x(t) is the input to an ideal bandpass ﬁlter having the frequency response characteristic shown in Figure P2.38. Determine the totalnoise power at the output of the ﬁlter. FIGURE P2.38 2.39 A lowpass Gaussian stochastic process X(t) has a power spectral density S(f)=⎦braceleftbigg N0|f|<B 0 otherwise Determine the power spectral density and the autocorrelation function of Y(t)=X2(t). 2.40 The covariance matrix of three random variables X1,X2, and X3is ⎡ ⎣C110 C13 0 C220 C310 C33⎤ ⎦ The linear transformation Y=AXis made where A=⎡ ⎣100 020101⎤ ⎦ Determine the covariance matrix of Y. 2.41 LetX(t) be a stationary real normal process with zero mean. Let a new process Y(t)b e deﬁned by Y(t)=X2(t) Determine the autocorrelation function of Y(t) in terms of the autocorrelation function of X(t).Hint: Use the result on Gaussian variables derived in Problem 2.27. 2.42 For the Nakagami PDF, given by Equation 2.3–67, deﬁne the normalized random variable X=R/√ /Omega1. Determine the PDF of X. 2.43 The input X(t) in the circuit shown in Figure P2.43 is a stochastic process with E[X(t)]=0 andRX(τ)=σ2δ(τ); i.e., X(t) is a white noise process. a. Determine the spectral density SY(f). b. Determine RY(τ) and E[Y2(t)]. Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 91 FIGURE P2.43 2.44 Demonstrate the validity of Equation 2.8–6. 2.45 Use the Chernoff bound to show that Q(x)≤e−x2/2. 2.46 Determine the mean, the autocorrelation sequence, and the power density spectrum of the output of a system with unit sample response h(n)=⎧ ⎪⎪⎨ ⎪⎪⎩1 n=0 −2 n=1 1 n=2 0 otherwise when the input x(n) is a white noise process with variance σ2 x. 2.47 The autocorrelation sequence of a discrete-time stochastic process is R(k)=⎦parenleftbig1 2⎦parenrightbig|k|. Determine its power density spectrum. 2.48 A discrete-time stochastic process X(n)≡X(nT) is obtained by periodic sampling of a continuous-time zero-mean stationary process X(t), where Tis the sampling interval; i.e., fs=1/Tis the sampling rate. a. Determine the relationship between the autocorrelation function of X(t) and the auto- correlation sequence of X(n). b. Express the power density spectrum of X(n) in terms of the power density spectrum of the process X(t). c. Determine the conditions under which the power density spectrum of X(n) is equal to the power density spectrum of X(t). 2.49 The random process V(t) is deﬁned as V(t)=Xcos 2πfct−Ysin 2πfct where XandYare random variables. Show that V(t) is wide-sense stationary if and only ifE(X)=E(Y)=0,E(X2)=E(Y2), and E(XY)=0. 2.50 Consider a band-limited zero-mean stationary stochastic process X(t) with power density spectrum SX(f)=⎦braceleftbigg 1 |f|≤W 0 otherwise X(t) is sampled at a rate fs=1/Tto yield a discrete-time process X(n)≡X(nT). a. Determine the expression for the autocorrelation sequence of X(n). b. Determine the minimum value of Tthat results in a white (spectrally ﬂat) sequence. Proakis-27466 book September 25, 2007 13:9 92 Digital Communications c. Repeat (b) if the power density spectrum of X(t)i s SX(f)=⎦braceleftbigg 1−|f|/W |f|≤W 0 otherwise 2.51 Show that the functions fk(t)=sinc⎦bracketleftbigg 2W⎦parenleftbigg t−k 2W⎦parenrightbigg⎦bracketrightbigg , k=0,±1,±2,... are orthogonal over the real line, i.e., ⎦integraldisplay∞ −∞fk(t)fj(t)dt=⎦braceleftbigg 1/2Wk =j 0 otherwise Therefore, the sampling theorem reconstruction formula may be viewed as a series expan- sion of the band-limited signal s(t), where the weights are samples of s(t) and the {fk(t)} are the set of orthogonal functions used in the series expansion. 2.52 The noise equivalent bandwidth of a system is deﬁned as Beq=1 G⎦integraldisplay∞ 0|H(f)|2df where G=max|H(f)|2. Using this deﬁnition, determine the noise equivalent bandwidth of the ideal bandpass ﬁlter shown in Figure P2.38 and the low-pass system shown inFigure P2.43. 2.53 Suppose that N(t) is a zero-mean stationary narrowband process. The autocorrelation function of the equivalent lowpass process Z(t)=X(t)+jY(t) is deﬁned as R Z(τ)=E⎦bracketleftbig Z∗(t)Z(t+τ)⎦bracketrightbig a. Show that E[Z(t)Z(t+τ)]=0 b. Suppose Rz(τ)=N0δ(τ), and let V=⎦integraldisplayT 0Z(t)dt Determine E⎦bracketleftbig V2⎦bracketrightbig and E⎦bracketleftbig |V|2⎦bracketrightbig . 2.54 Determine the autocorrelation function of the stochastic process X(t)=Asin(2πfct+/Theta1) where fcis a constant and /Theta1is a uniformly distributed phase, i.e., p(θ)=1 2π, 0≤θ≤2π Proakis-27466 book September 25, 2007 13:9 Chapter Two: Deterministic and Random Signal Analysis 93 2.55 LetZ(t)=X(t)+jY(t) be a complex random process, where X(t) and Y(t) are real- valued, independent, zero-mean, and jointly stationary Gaussian random processes. Weassume that X(t) and Y(t) are both band-limited processes with a bandwidth of Wand a ﬂat spectral density within their bandwidth, i.e., SX(f)=SY(f)=⎦braceleftbigg N0|f|≤W 0 otherwise 1. Find E[Z(t)] and RZ(t+τ,t), and show that Z(t) is WSS. 2. Find the power spectral density of Z(t). 3. Assume φ1(t),φ2(t),...,φ n(t) are orthonormal, i.e., ⎦integraldisplay∞ −∞φj(t)φ∗ k(t)dt=⎦braceleftbigg 1 j=k 0 otherwise and all φj(t)’s are band-limited to [ −W,W]. Deﬁne random variables Zjas the pro- jections of Z(t) on the φj(t)’s, i.e., Zj=⎦integraldisplay∞ −∞Z(t)φ∗ j(t)dt, j=1,2,..., n Determine E[Zj] and E[ZjZ∗ k] and conclude that the Zj’s are iid zero-mean Gaussian random variables. Find their common variance. 4. Let Zj=Zjr+jZji, where ZjrandZjidenote the real and imaginary parts, respec- tively, of Zj. Comment on the joint probability distribution of the 2 nrandom variables (Z1r,Z1i,Z2r,Z2i,..., Znr,Zni) 5. Let us deﬁne ˆZ(t)=Z(t)−n⎦summationdisplay j=1Zjφj(t) to be the error in expansion of Z(t) as a linear combination of φj(t)’s. Show that E[ˆZ(t)Z∗ k]=0 for all k=1,2,..., n. In other words, show that the error ˆZ(t) and all theZk’s are uncorrelated. Can you say ˆZ(t) and the Zk’s are independent? 2.56 LetX(t) denote a (real, zero-mean, WSS) bandpass process with autocorrelation function RX(τ) and power spectral density SX(f), where SX(0)=0, and let ˆX(t) denote the Hilbert transform of X(t). Then ˆX(t) can be viewed as the output of a ﬁlter, with impulse response1 πtand transfer function −jsgn( f), whose input is X(t). Recall that when X(t) passes through a system with transfer function H(f) and the output is Y(t), we have SY(f)=SX(f)|H(f)|2andSXY(f)=SX(f)H∗(f). 1. Prove that RˆX(τ)=RX(τ). 2. Prove that RXˆX(τ)=− ˆRX(τ) 3. If Z(t)=X(t)+jˆX(t), determine SZ(f). 4. Deﬁne Xl(t)=Z(t)e−j2πf0t. Show that Xl(t) is a lowpass WSS random process, and determine SXl(f). From the expression for SXl(f), derive an expression for RXl(τ). Proakis-27466 book September 25, 2007 13:9 94 Digital Communications 2.57 A noise process has a power spectral density given by Sn(f)=⎦braceleftbigg 10−8⎦parenleftbig 1−|f| 108⎦parenrightbig |f|<108 0 |f|>108 This noise is passed through an ideal bandpass ﬁlter with a bandwidth of 2 MHz centered at 50 MHz.1. Find the power content of the output process.2. Write the output process in terms of the in-phase and quadrature components, and ﬁnd the power in each component. Assume f 0=50 MHz. 3. Find the power spectral density of the in-phase and quadrature components.4. Now assume that the ﬁlter is not an ideal ﬁlter and is described by |H(f)| 2=⎦braceleftbigg|f| 106−49 49 MHz <|f|<51 MHz 0 otherwise Repeat parts 1, 2, and 3 with this assumption.