Chapter 2 (Discrete Random Variables): Joint PMFs of Multiple Random Variables (多个随机变量的联合分布列)

本文为 $Introduction$ $to$ $Probability$ 的读书笔记

Joint PMF

• Consider two discrete random variables $X$ and $Y$ associated with the same experiment. The probabilities of the values that $X$ and $Y$ can take are captured by the joint PMF of $X$ and $Y$, denoted $p_{X,Y}$.
  $$p_{X,Y}(x,y)=P(X=x,Y=y)$$
• In fact, we can calculate the PMFs of $X$ and $Y$ by using the formulas
  $$p_X(x)=\sum _y{p}_{X,Y}(x,y),p_Y(y)=\sum _x{p}_{X,Y}(x,y)$$We sometimes refer to $p_X$ and $p_Y$ as the marginal PMFs (边缘分布列), to distinguish them from the joint PMF.

---

Problem 26. PMF of the minimum of several random variables.
On a given day. your golf score takes values from the range 101 to 110. with probability 0.1, independent of other days. Determined to improve your score, you decide to play on three different days and declare as your score the minimum $X$ of the scores $X_1,X_2$, and $X_3$ on the different days. Calculate the PMF of $X$.

SOLUTION

• We have $P(X>100)=1$ and for $k=101,...,110$,
  
  It follows that
  

Functions of Multiple Random Variables

• A function $Z=g(X,Y)$ of the random variables $X$ and $Y$ defines another random variable. Its PMF can be calculated from the joint PMF $p_{X,Y}$ according to
  p Z ( z ) = ∑ { ( x , y ) ∣ g ( x , y ) = z } p X , Y ( x , y ) p_Z(z)=\sum_{\{(x,y)|g(x,y)=z\}}p_{X,Y}(x,y) pZ​(z)={ (x,y)∣g(x,y)=z}∑​pX,Y​(x,y)
• Furthermore. the expected value rule for functions naturally extends and takes the form
  $$E[g(X,Y)]=\sum _x\sum _{y}g(x,y)p_{X,Y}(x,y)$$In the special case where $g$ is linear and of the form $aX+bY+c$, where $a,b$, and $c$ are given scalars, we have
  $$E[aX+bY+c]=aE[X]+bE[Y]+c$$

More than Two Random Variables

• The joint PMF of three random variables $X,Y$, and $Z$ is defined in analogy with the above as
  $$p_{X,Y,Z}(x,y,z)=P(X=x,Y=y,Z=z)$$for all possible triplets of numerical values $(x,y,z)$. Corresponding marginal PMFs are analogously obtained by equations such as
  $$p_{X,Y}(x,y)=\sum _z{p}_{X,Y,Z}(x,y,z)$$and$$p_X(x)=\sum _y\sum _z{p}_{X,Y,Z}(x,y,z)$$
• The expected value rule for functions is given by
  $$E[g(X,Y,Z)]=\sum _x\sum _y\sum _{z}g(x,y,z)p_{X,Y,Z}(x,y,z)$$and if $g$ is linear and has the form $aX+bY+cZ+d$, then
  $$E[aX+bY+cZ+d]=aE[X]+bE[Y]+cE[Z]+d$$

---

Example 2.10. Mean of the Binomial
Your probability class has 300 students and each student has probability $1/3$ of getting an A, independent of any other student. What is the mean of $X$, the number of students that get an A?

• Let
  
• Thus $X_1,X_2,.....,X_n$ are Bernoulli random variables with common mean $p=1/3$. Their sum
  $$X=X_1+X_2+\cdot \cdot \cdot +X_n$$is the number of students that get an A. Since $X$ is the number of “successes”’ in $n$ independent trials, it is a binomial random variable with parameters $n$ and $p$.
• Using the linearity of $X$ as a function of the $X_i$, we have
  $$E[X]=\sum _{i=1}^{300}E[X_i]=\sum _{i=1}^{300}\frac{1}{3}=100$$If we repeat this calculation for a general number of students $n$ and probability of A equal to $p$, we obtain
  $$E[X]=\sum _{i=1}^{n}E[X_i]=\sum _{i=1}^{n}p=np$$

---

Example 2.11. The Hat Problem.
Suppose that $n$ people throw their hats in a box and then each picks one hat at random. (Each hat can be picked by only one person, and each assignment of hats to persons is equally likely.) What is the expected value of $X$, the number of people that get back their own hat?

• For the $i$th person. we introduce a random variable $X_i$ that takes the value 1 if the person selects his/her own hat. and takes the value 0 otherwise. Since $P(X_i=1)=1/n$ and $P(X_i=0)=1-1/n$, the mean of $X_i$ is $1/n$.
• We now have
  $$X=X_1+X_2+\cdot \cdot \cdot +X_n$$so that
  $$E[X]=E[X_1]+E[X_2]+\cdot \cdot \cdot +E[X_n]=n\cdot \frac{1}{n}=1$$

---

Problem 27. The multinomial distribution.
A die with $r$ faces, numbered $1,....r$. is rolled a fixed number of times $n$. The probability that the $i$th face comes up on any one roll is denoted $p_i$ , and the results of different rolls are assumed independent. Let $X_i$ be the number of times that the $i$th face comes up. Find $E[X_i{X}_j]$ for $i\ne j.$

SOLUTION

• Let $Y_{i,k}$ (or $Y_{j,k}$) be the Bernoulli random variable that takes the value 1 if face $i$ (respectively, $j$) comes up on the $k$th roll. and the value 0 otherwise. Note that $Y_{i,k}{Y}_{j,k}=0$, and that for $l\ne k$, $Y_{i,k}$ and $Y_{i,l}$ are independent, so that $E[Y_{i,k},Y_{j,l}]=p_i{p}_j$. Therefore,
  $$\begin{array}{rl}E[X_i{X}_j]& =E[(Y_{i,1}+...+Y_{i,n})(Y_{j,1}+...+Y_{j,n})]\\ & =n(n-1)E[Y_{i,1},Y_{j,2}]\\ & =n(n-1)p_i{p}_j\end{array}$$

---

Problem 29. The incIusion-exclusion formula. (容斥不等式)
Let $A_1,A_2,...,A_n$ be events. Let S 1 = { i ∣ 1 ≤ i ≤ n } S_1 = \{ i|1 \leq i\leq n\} S1​={ i∣1≤i≤n} , S 2 = { ( i 1 , i 2 ) ∣ 1 ≤ i 1 < i 2 ≤ n } S_2 = \{ (i_1 , i_2 ) |1\leq i_1 < i_2\leq n\} S2​={ (i1​,i2​)∣1≤i1​<i2​≤n}，and more generally, let $S_m$ be the set of all $m$-tuples $(i_1,...,i_m)$ of indices that satisfy $1\le i_1<i_2<...<i_m\le n$. Show that

• Hint: Let $X_i$ be a binary random variable which is equal to 1 when $A_i$ occurs, and equal to 0 otherwise. Relate the event of interest to the random variable $(1-X_1)(1-X_2)...(1-X_n)$

SOLUTION

• Let us express the event $B={\cup }_{k=1}^n{A}_k$ in terms of the random variables $X_1,...,X_n$. The event $B^C$ occurs when $Y=(1-X_1)(1-X_2)...(1-X_n)$ is equal to $1$.
  $$P(B^C)=P(Y=1)=E[Y]$$Therefore,
  $$\begin{array}{rl}P(B)& =1-E[(1-X_1)(1-X_2)...(1-X_n)]\\ & =E[X_1+...+X_n]-E[\sum _{(i_1,i_2)\in S_2}{X}_{i_1}{X}_{i_2}]+...+(-1{)}^{n-1}E[X_1...X_n]\end{array}$$We note that
  $$\begin{gathered} E[X_i]=P(A_i),E[X_{i_1}{X}_{i_2}]=P(A_{i_1}\cap A_{i_2}) \\ E[X_{i_1}{X}_{i_2}{X}_{i_3}]=P(A_{i_1}\cap A_{i_2}\cap A_{i_3}),E[X_1{X}_2...X_n]=P({\cap }_{k=1}^n{A}_k) \end{gathered}$$etc., from which the desired formula follows.

---

Problem 30.
Alvin’s database of friends contains $n$ entries, but due to a software glitch, the addresses correspond to the names in a totally random fashion. Alvin writes a holiday card to each of his friends and sends it to the (software-corrupted) address. What is the probability that at least one of his friends will get the correct card?

• Hint: Use the inclusion-exclusion formula.

SOLUTION

• Let $A_k$ be the event that the $k$th card is sent to the correct address. We have for any $k,j,i$,
  etc., and
  
• Applying the inclusion-exclusion formula, we obtain the desired probability
  
  When $n$ is large, this probability can be approximated by $1-e^{-1}$.