Introducing a method for visualizing feature maps and kernel weights

Recommended visualization tool TensorBoard: you can view the data flow of the entire calculation graph, save loss information, accuracy information, etc. during the retraining process.

Learning video:

Use pytorch to view the intermediate layer feature matrix and convolution kernel parameters_bilibili_bilibili

Code download:

deep-learning-for-image-processing/pytorch_classification/analyze_weights_featuremap at master · WZMIAOMIAO/deep-learning-for-image-processing · GitHub

1. Required documents

The AlexNet.pth and resNet34.pth files were obtained through previous training

2. Operation steps

AlexNet’s middle layer feature matrix

1. Set a breakpoint at out_put in the analyze_feature_map.py file and debug it to view the information printed by the print model.

2. Print two layer structures: the first is features, and the second is classifier, corresponding to the layer structure defined in the alexnet_model.py file, as shown in the following figure:

3. Set a breakpoint in the for name, module in self.features.named_children(): line of the alexnet_model.py file and run it step by step

Get name = 0 and convolution layer conv 2d, and so on.

4. Let the program then run to the for loop

Check out_put, which is a list. There are three layers in total, corresponding to the output feature matrices of the first, second, and third convolutional layers.

5. Let the program finish executing

(1) The output is the feature map of the first 12 channels of the feature matrix output by the first convolution layer.

We can understand some of the information that the convolutional layer focuses on through the brightness of the feature map. The higher the brightness, the more interesting the convolutional layer is.

The original picture is as follows:

(2) Feature matrix output by convolutional layer 2: The level of abstraction is getting higher and higher, and some convolution kernels do not play a role.

The feature matrix output by convolutional layer three

(3) The color after removing cmap='gray' is blue-green

(4) If you want to see more information, make modifications during the forward propagation process of alexnet_model.py

If you want to see the image of the fully connected layer, you must also pass the input image through the features layer structure and then through the fully connected layer to view it.

Intermediate layer feature matrix of resnet34

1. Modify the code and set a breakpoint for debugging at the figure below

You can see the layer structure of resnet in the terminal

2. The running results are as shown in the figure

Obviously resnet learned more information than alexnet

There are two reasons: resnet is indeed better than alexnet

resnet uses the transfer learning method, and the pre-training data set is trained using the imagenet data set.

3. The feature matrix output by layer1 is obviously much better than alexnet. Each feature layer has output and is useful.

AlexNet’s convolution kernel parameters

1. Open the analyze_kernel_weight.py file

Here you can directly load the training weights through the torch.load function without instantiating the model, because after loading through torch.load, it is a dictionary type, its key represents the name of each layer structure, and the corresponding value is each layer training information

2. Obtain the dictionary of all trainable parameters in the model through the model.state_dict function, and then obtain the names of all layer structures with parameters through the keys method

Run single step and look at weights_keys

As shown in the figure below, weights_keys is an ordered set of keys, which are saved in the order of the forward propagation process.

Naming rules:

Only convolutional layers such as feature0, feature3, feature6, feature8, and feature10 have training parameters.

There is no activation function for activation function and max pooling downsampling.

3. Next traverse weights_keys

The model.state_dict function obtains the dictionary information of all trainable parameters in the model, passes in the corresponding key to obtain the parameter information, and then uses the numpy method to convert the weight information into numpy format for easy analysis

Note: The order of convolution kernel channels is

kernel_number is the number of convolution kernels, corresponding to the depth of the output feature matrix.

kernel_channel convolution kernel depth, corresponding to the depth of the input feature matrix

kernel_height, kernel_width, the height and width of the convolution kernel

4. Get information

    # k = weight_t[0, :, :, :]  # 通过切片的方式获得信息

    # calculate mean, std, min, max    对所有卷积核的信息进行计算
    weight_mean = weight_t.mean()      #均值
    weight_std = weight_t.std(ddof=1)  #标准差
    weight_min = weight_t.min()        #最小值
    weight_max = weight_t.max()        #最大值

Distribution of convolution kernel values corresponding to convolution layer one