Author: Zen and the Art of Computer Programming
Probabilistic Graphical Model (PGM) is an important tool in modern statistical learning, which models complex systems by describing the dependencies and probability distributions between variables. A probabilistic graphical model consists of two parts: a probabilistic model, which defines the joint probability distribution between variables, and a structural model, which defines possible causal influences between variables. In the field of deep learning, PGM is widely used to represent the probabilistic dependencies in the data generation process, and can easily represent various complex structures.
This article will briefly introduce the probabilistic graphical model and the mathematical knowledge behind it, combined with some practical cases, to provide readers with the relevant background knowledge and methodology of the probabilistic graphical model. I hope it can help readers understand and use probabilistic graphical models more deeply.
2. Explanation of basic concepts and terms
(1) random variable
A variable is called a random variable (Random Variable), its value is taken from a certain probability distribution. For example, someone's height, weight, IQ, marital status are all random variables. A probability distribution refers to the randomness to which a variable is subject. Generally speaking, random variables can be divided into discrete random variables and continuous random variables.
discrete random variable
If the value of a random variable X can be enumerated into a finite number of elements, it is called a discrete random variable. The probability distribution of a discrete random variable is usually expressed by a probability mass function (Probability Mass Function, PMF), that is, when a random variable X=x is given, its probability P(X=x). For example, the result X of tossing a coin, X may be positive or negative, then the PMF of X is: $$ \begin{align } p(X=x_i)= \begin{cases} p,& x_i=H\ 1 -p,& x_i=T \end{cases}\quad (i=1,2),