Chapter 4(Further Topics on Random Variables):Sum of a Random Number of Independent Random Variables

本文为 $Introduction$ $to$ $Probability$ 的读书笔记

Sum of a Random Number of Independent Random Variables

• We consider the sum
  $$Y=X_1+...+X_N$$where $N$ is a random variable that takes nonnegative integer values, and $X_1,X_2....$ are identically distributed random variables. (If $N=0$, we let $Y=0$.) We assume that $N,X_1,X_2,...$ are independent, meaning that any finite subcollection of these random variables are independent.
• Let us denote by $E[X]$ and $var(X)$ the common mean and variance, respectively, of the $X_i$. We wish to derive formulas for the mean, variance, and the transform of $Y$.

$E[Y]$

• The method that we follow is to first condition on the event { N = n } \{N=n\} { N=n}, which brings us to the more familiar case of a $fixed$ number of random variables.
  $$E[Y|N=n]=E[X_1+...+X_N|N=n]=E[X_1+...+X_N]=nE[X]$$
• This is true for every nonnegative integer $n$, so
  $$E[Y|N]=NE[X]$$
• Using the law of iterated expectations, we obtain
  $$E[Y]=E[E[Y|N]]=E[NE[X]]=E[N]E[X]$$

$var(Y)$

• Similarly,
  $$var(Y|N=n)=var(X_1+...+X_N|N=n)=var(X_1+...+X_N)=nvar(X)$$
• Since this is true for every nonnegative integer $n$,
  $$var(Y|N)=Nvar(X)$$
• We now use the law of total variance to obtain
  $$var(Y)=E[var(Y|N)]+var(E[Y|N])=E[N]var(X)+(E[X]{)}^{2}var(N)$$

$M_Y(s)$

• The calculation of the transform proceeds along similar lines.
  $$\begin{gathered} E[e^{sY}|N=n]=E[e^{s{X}_1}...e^{s{X}_N}|N=n]=E[e^{s{X}_1}...e^{s{X}_N}] \\ =E[e^{s{X}_1}]...E[e^{s{X}_N}]=(M_X(s){)}^n \end{gathered}$$Using the law of iterated expectations, the (unconditional) transform associated with $Y$ is
  $$M_Y(s)=E[e^{sY}]=E[E[e^{sY}|N]]=E[(M_X(s){)}^N]=\sum _{n=0}^{\infty }(M_X(s){)}^n{p}_N(n)$$
• Using the observation
  $$(M_X(s){)}^n=e^{log(M_X(s){)}^n}=e^{nlog{M}_X(s)}$$we have
  $$M_Y(s)=\sum _{n=0}^{\infty }{e}^{nlog{M}_X(s)}{p}_N(n)$$
• Comparing with the formula
  $$M_N(s)=\sum _{n=0}^{\infty }{e}^{sn}{p}_N(n)$$we see that
  $$M_Y(s)=M_N(log{M}_X(s))$$

---

Example 4.35. Sum of a Geometric Number of Independent Exponential Random Variables.
Jane visits a number of bookstores, looking for $Great$ $Expectations$. Any given bookstore carries the book with probability $p$, independent of the others. In a typical bookstore visited, Jane spends a random amount of time, exponentially distributed with parameter $\lambda$，until she either finds the book or she determines that the bookstore does not carry it. We assume that Jane will keep visiting bookstores until she buys the book and that the time spent in each is independent of everything else. We wish to find the mean, variance, and PDF of the total time spent in bookstores.

SOLUTION

• The total number $N$ of bookstores visited is geometrically distributed with parameter $p$. Hence, the total time $Y$ spent in bookstores is the sum of a geometrically distributed number $N$ of independent exponential random variables $X_1,X_2,....$ We have
  $$E[Y]=E[N]E[X]=\frac{1}{p}\cdot \frac{1}{\lambda }$$
• Using the formulas for the variance of geometric and exponential random variables, we also obtain
  $$var(Y)=E[N]var(X)+(E[X]{)}^{2}var(N)=\frac{1}{p}\cdot \frac{1}{{\lambda }^2}+\frac{1}{{\lambda }^2}\cdot \frac{1-p}{p^2}=\frac{1}{{\lambda }^2{p}^2}$$
• In order to find the transform $M_Y(s)$, let us recall that
  $$M_X(s)=\frac{\lambda }{\lambda -s},M_N(s)=\frac{p{e}^s}{1-(1-p)e^s}$$Then, $M_Y(s)$ is found by starting with $M_N(s)$ and replacing each occurrence of $e^s$ with $M_X(s)$. This yields
  $$M_Y(s)=\frac{p\frac{\lambda }{\lambda -s}}{1-(1-p)\frac{\lambda }{\lambda -s}}=\frac{p\lambda }{p\lambda -s}$$
• We recognize this as the transform associated with an exponentially distributed random variable with parameter $p\lambda$, and therefore,
  $$f_Y(y)=p\lambda e^{-p\lambda y},y\ge 0$$This result can be surprising because the sum of a $fixed$ number $n$ of independent exponential random variables is not exponentially distributed.

---

Example 4.36. Sum of a Geometric Number of Independent Geometric Random Variables.

• This example is a discrete counterpart of the preceding one. We let $N$ be geometrically distributed with parameter $p$. We also let each random variable $X_i$ be geometrically distributed with parameter $q$. We assume that all of these random variables are independent. Let $Y=X_1+\cdot \cdot \cdot +X_N$. We have
  $$\begin{gathered} M_N(s)=\frac{p{e}^s}{1-(1-p)e^s},M_X(s)=\frac{q{e}^s}{1-(1-q)e^s} \\ \therefore M_Y(s)=\frac{p{M}_X(s)}{1-(1-p)M_X(s)}=\frac{pq{e}^s}{1-(1-pq)e^s} \end{gathered}$$
• We conclude that $Y$ is geometrically distributed, with parameter $pq$.

---

Problem 43.
A motorist goes through 4 lights, each of which is found to be red with probability $1/2$. The waiting times at each light are modeled as independent normal random variables with mean $1$ minute and standard deviation $1/2$ minute. Let $X$ be the total waiting time at the red lights.

• (a) Use the total probability theorem to find the PDF and the transform associated with $X$, and the probability that $X$ exceeds 4 minutes. Is $X$ normal?
• (b) Find the transform associated with $X$ by viewing $X$ as a sum of a random number of random variables.

SOLUTION

• (a) The conditional PDF of $X$ given that $k$ lights are red, is normal with mean $k$ minutes and standard deviation $(1/2)\sqrt{k}$. Thus, $X$ is a mixture of normal random variables and the transform associated with its (unconditional) PDF is the corresponding mixture of the transforms associated with the (conditional) normal PDFs. However, $X$ is not
  normal, because a mixture of normal PDFs need not be normal. The probability $P(X>4|klightsarered)$ can be computed from the normal tables for each $k$, and
  $$P(X>4)=\sum _{k=0}^{4}P(klightsarered)P(X>4|klightsarered)$$
• (b) Let $K$ be the number of traffic lights that are found to be red. We can view $X$ as the sum of $K$ independent normal random variables. Thus the transform associated with $X$ can be found by replacing in the binomial transform $M_K(s)=(1/2+(1/2)e^s{)}^4$ the occurrence of $e^s$ by the normal transform corresponding to $\mu =1$ and $\sigma =1/2$. Thus
  $$M_X(s)=(\frac{1}{2}+\frac{1}{2}(e^{\frac{(1/2{)}^2{s}^2}{2}+s}){)}^4$$