Chapter 4 (Further Topics on Random Variables): Derived Distributions (随机变量函数的概率密度函数)

本文为 $Introduction$ $to$ $Probability$ 的读书笔记

Derived Distributions

---

Example 4.3.

• Let $Y=g(X)=X^2$, where $X$ is a random variable with known PDF. For any $y\ge 0$, we have
  $$\begin{array}{rl}{F}_Y(y)& =P(Y\le y)\\ & =P(X^2\le y)\\ & =P(-\sqrt{y}\le X\le \sqrt{y})\\ & =F_X(\sqrt{y})-F_X(-\sqrt{y})\end{array}$$and therefore, by differentiating and using the chain rule,
  $$f_Y(y)=\frac{1}{2\sqrt{y}}{f}_X(\sqrt{y})+\frac{1}{2\sqrt{y}}{f}_X(-\sqrt{y}),y\ge 0$$

---

Problem 1.
If $X$ is a random variable that is uniformly distributed between $-1$ and $1$. find the PDF of $-ln|X|$.

SOLUTION

• Let $Y=-ln|X|$. We have, for $y\ge 0$,
  $$F_Y(y)=P(Y\le y)=P(ln|X|\ge -y)=P(X\ge e^{-y})+P(X\le -e^{-y})=1-e^{-y}$$and therefore by differentiation
  $$f_Y(y)=e^{-y}fory\ge 0$$so $Y$ is an exponential random variable with parameter $1$.
• This exercise provides a method for simulating an exponential random variable using a sample of a uniform random variable.

The Linear Case

• We now focus on the important special case where $Y$ is a linear function of $X$.
  
  

To verify this formula, divide the problem into two cases: $a>0$ and $a<0$.

---

Example 4.5. A Linear Function of a Normal Random Variable is Normal.

• Suppose that $X$ is a normal random variable with mean $\mu$ and variance $\sigma$, and let $Y=aX+b$, where $a$ and $b$ are scalars, with $a\ne 0$. We have
  $$f_X(x)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-(x-\mu {)}^2/2{\sigma }^2}$$Therefore,
  $$f_Y(y)=\frac{1}{\sqrt{2\pi }|a|\sigma }{e}^{-\frac{(\frac{y-b}{a}-\mu {)}^2}{2{\sigma }^2}}=\frac{1}{\sqrt{2\pi }|a|\sigma }{e}^{-\frac{(y-b-a\mu {)}^2}{2a^2{\sigma }^2}}$$We recognize this as a normal PDF with mean $a\mu +b$ and variance $a^2\sigma$. In particular, $Y$ is a normal random variable.

The Monotonic Case

单调函数

• The calculation and the formula for the linear case can be generalized to the case where $g$ is a monotonic function.
• Let $X$ be a continuous random variable and suppose that its range is contained in a certain interval $I$. We consider the random variable $Y=g(X)$, and assume that $g$ is strictly monotonic over the interval $I$. Furtherrnore. we assume that the function $g$ is differentiable.
  (An important fact is that for a strictly monotonic function $g$, there exists an inverse function $h=g^{-1}$.)
  

Functions of Two Random Variables

• The two-step procedure that first calculates the CDF and then differentiates to obtain the PDF also applies to functions of more than one random variable.

---

Example 4.8.
Let $X$ and $Y$ be independent random variables that are uniformly distributed on the interval $[0,1]$. What is the PDF of the random variable $Z=Y/X$?

SOLUTION

• We consider separately the cases $0\le z\le 1$ and $z>1$. As shown in Fig. 4.5, we have
  
  
• By differentiating, we obtain
  

---

Example 4.9.
Romeo and Juliet have a date at a given time, and each, independently, will be late by an amount of time that is exponentially distributed with parameter $\lambda$. What is the PDF of the difference between their times of arrival?

SOLUTION

• Let us denote by $X$ and $Y$ the amounts by which Romeo and Juliet are late, respectively. We want to find the PDF of $Z=X-Y$, assuming that $X$ and $Y$ are independent and exponentially distributed with parameter $\lambda$. We will first calculate the CDF $F_Z(z)$ by considering separately the cases $z\ge 0$ and $z<0$ (see Fig. 4.6).
  
• For $z\ge 0$, we have (see the left side of Fig. 4.6)
  $$\begin{array}{rl}{F}_Z(z)& =P(X-Y\le z)\\ & =1-P(X-Y>z)\\ & =1-{\int }_0^{\infty }({\int }_{z+y}^{\infty }{f}_{X,Y}(x,y)dx)dy\\ & =1-{\int }_0^{\infty }\lambda e^{-\lambda y}({\int }_{z+y}^{\infty }\lambda e^{-\lambda x}dx)dy\\ & =1-{\int }_0^{\infty }\lambda e^{-\lambda y}{e}^{-\lambda (z+y)}dy\\ & =1-\frac{1}{2}{e}^{-\lambda z}\end{array}$$
• For the case $z<0$, we can use a similar calculation, but we can also argue using symmetry. Indeed, the symmetry of the situation implies that the random variables $Z=X-Y$ and $-Z=Y-X$ have the same distribution. We have
  $$F_Z(z)=P(Z\le z)=P(-Z\ge -z)=P(Z\ge -z)=1-F_Z(-z)$$With $z<0$, we have $-z>0$ and using the formula derived earlier,
  $$F_Z(z)=1-F_Z(-z)=1-(1-\frac{1}{2}{e}^{\lambda z})=\frac{1}{2}{e}^{\lambda z}$$
• Combining the two cases $z\ge 0$ and $z<0$, we obtain
  
• We now calculate the PDF of $Z$ by differentiating its CDF. We have
  or$$f_Z(z)=\frac{\lambda }{2}{e}^{-\lambda |z|}$$This is known as a two-sided exponential PDF (双边指数概率密度函数), also called the Laplace PDF (拉普拉斯概率密度函数).

---

Problem 11.
Use the convolution formula to establish that the sum of two independent Poisson random variables with parameters $\lambda$ and $\mu$, respectively, is Poisson with parameter $\lambda +\mu$.

SOLUTION

• The convolution of two Poisson PMFs is of the form
  $$\sum _{i=0}^k\frac{{\lambda }^i{e}^{-\lambda }}{i!}\cdot \frac{{\mu }^{k-i}{e}^{-\mu }}{(k-i)!}=e^{-(\lambda +\mu )}\sum _{i=0}^k\frac{{\lambda }^i{\mu }^{k-i}}{i!(k-i)!}$$We have
  $$(\lambda +\mu {)}^k=\sum _{i=0}^k\left(\begin{array}{c}k\\ i\end{array}\right){\lambda }^i{\mu }^{k-i}=\sum _{i=0}^k\frac{k!}{i!(k-i)!}{\lambda }^i{\mu }^{k-i}$$Thus, the desired PMF is
  $$\frac{e^{-(\lambda +\mu )}}{k!}\sum _{i=0}^k\left(\begin{array}{c}k\\ i\end{array}\right){\lambda }^i{\mu }^{k-i}=\frac{e^{-(\lambda +\mu )}}{k!}(\lambda +\mu {)}^k,$$which is a Poisson PMF with mean $\lambda +\mu$.

---

Problem 16. The polar coordinates of two independent normal random variables.
Let $X$ and $Y$ be independent standard normal random variables. The pair $(X,Y)$ can be described in polar coordinates in terms of random variables $R\ge 0$ and $\Theta \in [0,2\pi ]$, so that
$$X=R\cos \Theta ,Y=R\sin \Theta$$

• (a) Show that $\Theta$ is uniformly distributed in $[0,2\pi ]$, that $R$ has the PDF
  $$f_R(r)=r{e}^{-r^2/2},r\ge 0$$and that $R$ and $\Theta$ are independent.
  (The random variable $R$ is said to have a Rayleigh distribution. (瑞利分布))
• (b) Show that $R^2$ has an exponential distribution with parameter $1/2$.

[Note: Using the results in this problem, we see that samples of a normal random variable can be generated using samples of independent uniform and exponential random variables.]

SOLUTION

• (a) We first find the joint CDF of $R$ and $\Theta$. Fix some $r>0$ and some $\theta \in [0.2\pi ]$, and let $A$ be the set of points $(x,y)$ whose polar coordinates $(\overline{r},\overline{\theta })$ satisfy $0\le \overline{r}\le r$ and $0\le \overline{\theta }\le \theta$; note that the set $A$ is a sector of a circle of radius $r$. with angle $\theta$. We have
  $$\begin{array}{rl}{F}_{R,\Theta }(r,\theta )& =P(R\le r,\Theta \le \theta )=P((X,Y)\in A)\\ & =\frac{1}{2\pi }\int {\int }_{(x,y)\in A}{e}^{-\frac{x^2+y^2}{2}}dxdy\\ & =\frac{1}{2\pi }{\int }_0^{\theta }{\int }_0^r{e}^{-\frac{{\overline{r}}^2}{2}}\overline{r}d\overline{r}d\overline{\theta }\end{array}$$We then differentiate, to find that
  $$f_{R,\Theta }(r,\theta )=\frac{{\partial }^2{F}_{R,\Theta }(r,\theta )}{\partial r\partial \theta }=\frac{r}{2\pi }{e}^{-r^2/2},r\ge 0$$Thus,
  $$f_R(r)={\int }_0^{2\pi }{f}_{R,\Theta }(r,\theta )d\theta =r{e}^{-r^2/2},r\ge 0$$Furthermore,
  $$f_{\Theta |R}(\theta |r)=\frac{f_{R,\Theta }(r,\theta )}{f_R(r)}=\frac{1}{2\pi },\theta \in [0,2\pi ]$$Since the conditional PDF $f_{\Theta |R}(\theta |r)$ is unaffected by the value of the conditioning variable $R$. it follows that it is also equal to the unconditional PDF $f_{\Theta }$. In particular, $f_{\Theta |R}(\theta |r)=f_R(r)f_{\Theta }(\theta )$, so that $R$ and $\Theta$ are independent.
• (b) Let $t\ge 0$. We have
  $$P(R^2\ge t)=P(R\ge \sqrt{t})={\int }_{\sqrt{t}}^{\infty }r{e}^{-r^2/2}dr={\int }_{t/2}^{\infty }{e}^{-u}du=e^{-t/2}$$where we have used the change of variables $u=r^2/2$. By differentiating, we obtain
  $$f_{R^2}(t)=\frac{1}{2}{e}^{-t/2},t\ge 0$$

Sums of Independent Random Variables – Convolution (卷积)

• For some initial insight, we start by deriving a PMF formula for the case where $X$ and $Y$ are discrete. Let $Z=X+Y$, where $X$ and $Y$ are independent integer-valued random variables with PMFs $p_X$ and $p_Y$, respectively. Then, for any integer $z$,
  p Z ( z ) = P ( X + Y = z ) = ∑ { ( x , y ) ∣ x + y = z } P ( X = x , Y = y ) = ∑ x P ( X = x , Y = z − x ) = ∑ x p X ( x ) p Y ( z − x ) \begin{aligned}p_Z(z)&=P(X+Y=z) \\&=\sum_{\{(x,y)|x+y=z\}}P(X=x,Y=y) \\&=\sum_xP(X=x,Y=z-x) \\&=\sum_xp_X(x)p_Y(z-x) \end{aligned} pZ​(z)​=P(X+Y=z)={ (x,y)∣x+y=z}∑​P(X=x,Y=y)=x∑​P(X=x,Y=z−x)=x∑​pX​(x)pY​(z−x)​The resulting PMF $p_Z$ is called the convolution of the PMFs of $X$ and $Y$.
  

---

• Suppose now that $X$ and $Y$ are independent continuous random variables with PDFs $f_X$ and $f_Y$, respectively. We wish to find the PDF of $Z=X+Y$. Towards this goal, we will first find the joint PDF of $X$ and $Z$, and then integrate to find the PDF of $Z$.
  $$\begin{array}{rl}P(Z\le z|X=x)& =P(X+Y\le z|X=x)\\ & =P(x+Y\le z|X=x)\\ & =P(x+Y\le z)\\ & =P(Y\le z-x)\end{array}$$where the third equality follows from the independence of $X$ and $Y$. By differentiating both sides with respect to $z$, we see that $f_{Z|X}(z|x)=f_Y(z-x)$. Using the multiplication rule, we have
  $$f_{X,Z}(x,z)=f_X(x)f_{Z|X}(z|x)=f_X(x)f_Y(z-x)$$from which we finally obtain
  $$f_Z(z)={\int }_{-\infty }^{\infty }{f}_{X,Z}(x,z)dx={\int }_{-\infty }^{\infty }{f}_X(x)f_Y(z-x)dx$$
  

---

Example 4.10.

• The random variables $X$ and $Y$ are independent and uniformly distributed in the interval $[0,1]$. The PDF of $Z=X+Y$ is
  $$f_Z(z)={\int }_{-\infty }^{\infty }{f}_X(x)f_Y(z-x)dx$$
• The integrand $f_X(x)f_Y(z-x)$ is nonzero (and equal to 1) for $0\le x\le 1$ and $0\le z-x\le 1$. Combining these two inequalities, the integrand is nonzero for m a x { 0 , z − 1 } ≤ x ≤ m i n { 1 , z } max\{0, z - 1\}\leq x \leq min\{1, z\} max{ 0,z−1}≤x≤min{ 1,z}. Thus,
  

---

Example 4.11. The Sum of Two Independent Normal Random Variables is Normal.

• Let $X$ and $Y$ be independent normal random variables with means ${\mu }_x,{\mu }_y$, and variances ${\sigma }_x^2,{\sigma }_y^2$, respectively. and let $Z=X+Y$. We have
  f Z ( z ) = ∫ − ∞ ∞ f X ( x ) f Y ( z − x ) d x = ∫ − ∞ ∞ 1 2 π σ x e x p { − ( x − μ x ) 2 2 σ x 2 } 1 2 π σ y e x p { − ( z − x − μ y ) 2 2 σ y 2 } d x = 1 2 π ( σ x 2 + σ y 2 ) e x p { − ( z − μ x − μ y ) 2 2 ( σ x 2 + σ y 2 ) } \begin{aligned}f_Z(z)&=\int_{-\infty}^\infty f_X(x)f_Y(z-x)dx \\&=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}\sigma_x}exp\{-\frac{(x-\mu_x)^2}{2\sigma_x^2}\}\frac{1}{\sqrt{2\pi}\sigma_y}exp\{-\frac{(z-x-\mu_y)^2}{2\sigma_y^2}\}dx \\&=\frac{1}{\sqrt{2\pi(\sigma_x^2+\sigma_y^2)}}exp\{-\frac{(z-\mu_x-\mu_y)^2}{2(\sigma_x^2+\sigma_y^2)}\} \end{aligned} fZ​(z)​=∫−∞∞​fX​(x)fY​(z−x)dx=∫−∞∞​2π ​σx​1​exp{ −2σx2​(x−μx​)2​}2π ​σy​1​exp{ −2σy2​(z−x−μy​)2​}dx=2π(σx2​+σy2​) ​1​exp{ −2(σx2​+σy2​)(z−μx​−μy​)2​}​Given that scalar multiples of normal random variables are also normal, it follows that $aX+bY$ is also normal, for any nonzero $a$ and $b$.

---

Example 4.12. The Difference of Two Independent Random Variables.

• The convolution formula can also be used to find the PDF of $X-Y$, when $X$ and $Y$ are independent, by viewing $X-Y$ as the sum of $X$ and $-Y$. We observe that the PDF of $-Y$ is given by $f_{-Y}(y)=f_Y(-y)$, and obtain
  $$f_{X-Y}(z)={\int }_{-\infty }^{\infty }{f}_X(x)f_{-Y}(z-x)dx={\int }_{-\infty }^{\infty }{f}_X(x)f_Y(x-z)dx$$

---

Problem 12.
The random variables $X,Y,$ and $Z$ are independent and uniformly distributed between zero and one. Find the PDF of $X+Y+Z$.

SOLUTION

• Let $V=X+Y$. As in Example 4.10, the PDF of $V$ is
  
• Let $W=X+Y+Z=V+Z$. We convolve the PDFs $f_V$ and $f_Z$, to obtain
  $$f_W(w)=\int f_V(v)f_Z(w-v)dv$$We first need to determine the limits of the integration. We see that the integrand can be nonzero only if
  
• We consider three separate cases. If $w\le 1$, we have
  $$f_W(w)={\int }_0^w{f}_V(v)f_Z(w-v)dv={\int }_0^{w}vdv=\frac{w^2}{2}$$
• If $1\le w\le 2$, we have
  $$\begin{array}{rl}{f}_W(w)& ={\int }_{w-1}^w{f}_V(v)f_Z(w-v)dv\\ & ={\int }_{w-1}^{1}vdv+{\int }_1^w(2-v)dv\\ & =\frac{1}{2}-\frac{(w-1{)}^2}{2}-\frac{(w-2{)}^2}{2}+\frac{1}{2}\end{array}$$
• Finally, if $2\le w\le 3$, we have
  $$f_W(w)={\int }_{w-1}^2{f}_V(v)f_Z(w-v)dv={\int }_{w-1}^2(2-v)dv=\frac{(3-w{)}^2}{2}$$
• To summarize,
  

---

Problem 13.
Consider a PDF that is positive only within an interval $[a,b]$ and is symmetric around the mean $(a+b)/2$. Let $X$ and $Y$ be independent random variables that both have this PDF. Suppose that you have calculated the PDF of $X+Y$. How can you easily obtain the PDF of $X-Y$?

• We have $X-Y=X+Z-(a+b)$, where $Z=a+b-Y$ is distributed identically with $X$ and $Y$ . Thus, the PDF of $X+Z$ is the same as the PDF of $X+Y$, and the PDF of $X-Y$ is obtained by shifting the PDF of $X+Y$ to the left by $a+b$.

Graphical Calculation of Convolutions

• Consider two PDFs $f_X(t)$ and $f_Y(t)$. For a fixed value of $z$, the graphical evaluation of the convolution
  $$f_Z(z)={\int }_{-\infty }^{\infty }{f}_X(t)f_Y(z-t)dt$$consists of the following steps:
  • $(a)$ We plot $f_Y(z-t)$ as a function of $t$. This plot has the same shape as the plot of $f_Y(t)$ except that it is first “flipped”. and then shifted by an amount $z$. If $z>0$. this is a shift to the right, if $z<0$. this is a shift to the left.
  • $(b)$ We place the plots of $f_X(t)$ and $f_Y(z-t)$ on top of each other, and form their product.
  • $(c)$ We calculate the value of $f_Z(z)$ by calculating the integral of the product of these two plots.