Flow-based Generative Model for Generating Networks

When I read the key point paper recently, I found that the Flow-based generation network can be used to simulate and generate the real potential error probability distribution, thereby increasing the acquisition of Regression-based information and greatly improving the AP of the Regression-based method. In order to really understand this paper, I learned about Flow-based Generative Model in detail, and recorded it here. If there is something wrong, please give me advice.

Generator

First of all, we need to know that the Flow-based Generative Model is essentially a generator, and the generator G defines a probability density function $P_G$, all we have to do is let $P_G$Infinitely close to $P_{data}$That is, the true sample distribution. How to approach it? Here you need to understand the concept of maximum likelihood estimation.

The maximum likelihood estimation roughly means that we know a probability density function, and the internal parameters cannot be determined, but we can conduct a large number of experiments, substitute the experimental results into the function, and find the parameter with the highest probability, that is, determine the probability density function.

PG P_G here$P_G$is a probability density function of a random parameter, we start from $P_{data}$Draw samples xix^i from the true sample distribution$x^i$ , take it into$P_G(x^i)$,并max($log{P}_G(x^i)$), through continuous iteration can make$P_G$Getting closer to $P_{data}$. It turns out that this is probably the case. The following introduces the mathematical knowledge needed for the Flow-based Generative Model.

Math Background

1. Jacobian matrix
First, you need to understand the Jacobian matrix, known $x=f(z),f$ is a function of z, thenffThe Jacobian matrix of$f$$J_f$。

2. Determinant


The physical meaning of determinant can be understood as the area or volume of a polygon with rows as vectors.

We know two probability density functions $\pi \left(z\right),p\left(x\right)$ , these two probability density functions exist$x=f\left(z\right)$ such a changing relationship. According to the fact that the areas of the integrals of the corresponding parts of the two functions are equal, that is, the area of ​​the purple square in the above figure is equal to the area of ​​the rhombus on the right (although it does not look equal), the equation in the figure above can be obtained. The equation can be understood in this way, in$\Delta z,\Delta x$ interval to$\pi \left(z\right),p\left(x\right)$ is integrated, since x passes through$x=f\left(z\right)$ can be mapped to z, so the area of ​​the integral (here should be understood as the volume) can be guaranteed to remain unchanged, and it becomes only$\Delta z,The\; area\; of\; the\; \Delta x$ interval and$\pi \left(z\right),The\; density\; of\; p\left(x\right)$ . And because$\Delta z,\Delta x$ interval is small enough, in$\Delta z,The\; integral\; on\; the\; \Delta x$ interval can be directly equivalent to$\Delta z,The\; area\; of\; the\; \Delta x$ interval is multiplied by the density, and the area of ​​the rhombus on the right can be expressed by the absolute value of the determinant, so the equation in the above figure can be obtained.
Through a series of equivalent changes of the determinant in the above figure, we can finally obtain the equation in the yellow box in the above figure. That is, it is known that$x=f\left(z\right)$，$\pi \left(z\right),p\left(x\right)$ can be calculated according toffThe absolute values ​​of the determinants of the Jacobians of$f\; map\; to\; each\; other.$

Flow-based Generative

In the Flow-based model, we can use the neural network as G, $x^i$ is a sample from the real distribution. When G is reversible, the inverse of G and the Jacobian matrix can be obtained to obtain$p_G(x^i)$. Therefore, we train the inverse of G and use G for inference. The figure below is a demo example of the Flow-based Generative application.