1. Definition of Dropout
Dropout is only used when Traning, because dropout is similar to regularization.
Commonly used in the hidden layer of a multilayer perceptron.
In predict, it is not needed, because predict is a stable result.
2. Dropout is implemented from scratch
Matrix multiplication, high efficiency
2. Simple implementation of pytorch
refer to
https://www.bilibili.com/video/BV1Y5411c7aY?p=1