Deep feature visualization (pytorch+visdom)

1. Install the visualization tool visdom

pip install visdom

2. Install dependent libraries

pip install opencv-python

3. Open the visdom server and set the port number to 5123

python -m visdom.server -port=5123

The following interface appears, indicating that the visualization server is running successfully. 

Enter the corresponding IP address to view the visualization server that runs successfully. Among them, 10.106.14.226 is the IP address of the laboratory server. 

 4. Run the torch network to visualize deep features

        # python -m visdom.server -port=5123
        # 第一帧初始化可视化服务器
        if self._frame==1:
            self.visini(5123)
        
        # 读入图像序号
        imagenum=str(self._frame)
        imagenum=imagenum.zfill(4)
        import cv2
        image=cv2.imread('/data3/publicData/Datasets/OTB/OTB2015/Jogging/img/'+imagenum+'.jpg')
        image = cv2.resize(image, dsize=(320, 240), interpolation=cv2.INTER_NEAREST)
        # 注意cv2的图片是BGR，用visdom显示出来是RGB的，而且通道数在第一位。需要用图示方法进行转换。
        # ...代表剩下的维度们，也可以放在前面，代表前面的维度们。
        # 显示原图
        self.viz.image(image.transpose(2, 0, 1)[::-1,...],win='x1',opts={'title': 'source image'}) 


        # 显示特征        
        feat = x[0].mean(dim = 1)
        feat = feat.unsqueeze(dim=1)
        feat=torch.nn.functional.interpolate(feat, size = [image.shape[0], image.shape[1]], mode = 'bilinear')
        feat = (feat - feat.min())/(feat.max() - feat.min())
        self.viz.image(feat, win='x2',opts={'title': 'feat image'})

        
        # 特征图伪彩色
        featshow=feat.cpu().numpy()
        featshow = np.uint8(255 * featshow[0][0])
        featshow=cv2.resize(featshow, (image.shape[1], image.shape[0]))
        featshow = cv2.applyColorMap(featshow, cv2.COLORMAP_JET)
        self.viz.image(featshow.transpose(2, 0, 1)[::-1,...], win='x3',opts={'title': 'heatmap'})



        # 特征图与原图融合
        fusedImg = cv2.addWeighted(image, 0.7, featshow, 0.3, 0)
        self.viz.image(fusedImg.transpose(2, 0, 1)[::-1,...],win='x4',opts={'title': 'fused image'}) 

Among them, x[0] is the feature output by the first layer of FPN, and the self.visini function is:

    # 初始化visdom，端口号与启用的相对应5123
    def visini(self, portnum):
        # python -m visdom.server -port=5123
        from visdom import Visdom
        self.viz = Visdom(port = portnum)
        assert self.viz.check_connection()

5. Visualize results

x[0] feature visualization results

x[1] feature visualization results

x[2] feature visualization results

x[3] feature visualization results

x[4] feature visualization results