AttributeError: ‘NoneType’ object has no attribute ‘data’

参考资料

1. issue
2. pytorch: grad is None while training
3. pytorch冻结部分参数训练另一部分

问题描述

将别人模型替换成自己的模型，训练过程中出现错误param.grad.data.clamp_(-grad_clip, grad_clip) AttributeError: 'NoneType' object has no attribute 'data'

解决方法

上网查询这个问题，大部分回答都是模型中定义的某个层没有参与到前向传播，所以反向传播，计算loss时，grad is None。主要是要找到未参与计算的层，并注释掉。

这个问题困扰我很久，因为始终觉得没有多余的层。

下面的代码定义一个AttentionResNet模型。后来才发现罪魁祸首是self.encoder，这所以前面一直没有注意到它，是因为后面的层的定义用到self.encoder中的某个层或某几个层组合。

class AttentionResNet(nn.Module):
    
    def __init__(self, in_channels=3, out_channels=1, num_filters=32, encoder_depth=34, pretrained=True):
        super(AttentionResNet, self).__init__()
        self.in_channels  = in_channels
        self.out_channels = out_channels 
        self.num_filters = num_filters
         
        if encoder_depth == 34:
            self.encoder = torchvision.models.resnet34(pretrained=pretrained)
            bottom_channel_nr = 512
        elif encoder_depth == 101:
            self.encoder = torchvision.models.resnet101(pretrained=pretrained)
            bottom_channel_nr = 2048
        elif encoder_depth == 152:
            self.encoder = torchvision.models.resnet152(pretrained=pretrained)
            bottom_channel_nr = 2048
        else:
            raise NotImplementedError('only 34, 101, 152 version of Resnet are implemented')

        #attention module
        self.pool  = nn.MaxPool2d(2, 2)
        self.relu  = nn.ReLU(inplace=True)
        self.conv1 = nn.Sequential(self.encoder.conv1, self.encoder.bn1, self.encoder.relu, self.pool)
        self.conv2 = self.encoder.layer1
        self.conv3 = self.encoder.layer2
        self.conv4 = self.encoder.layer3
        self.conv5 = self.encoder.layer4  

        self.center = DecoderBlockV2(bottom_channel_nr,                        num_filters * 8 * 2, num_filters * 8)
        self.dec5   = DecoderBlockV2(bottom_channel_nr + num_filters * 8,      num_filters * 8 * 2, num_filters * 8)
        self.dec4   = DecoderBlockV2(bottom_channel_nr // 2 + num_filters * 8, num_filters * 8 * 2, num_filters * 8)
        self.dec3   = DecoderBlockV2(bottom_channel_nr // 4 + num_filters * 8, num_filters * 4 * 2, num_filters * 2)
        self.dec2   = DecoderBlockV2(bottom_channel_nr // 8 + num_filters * 2, num_filters * 2 * 2, num_filters * 2 * 2)
        self.dec1   = DecoderBlockV2(num_filters * 2 * 2,                      num_filters * 2 * 2, num_filters)
        
        self.attention_map = nn.Sequential(
            ConvRelu(num_filters, num_filters),
            nn.Conv2d(num_filters, 1, kernel_size=1)  
        )  



    def forward(self, x):        

        #attention module
        conv1 = self.conv1(x)
        conv2 = self.conv2(conv1)
        conv3 = self.conv3(conv2)
        conv4 = self.conv4(conv3)
        conv5 = self.conv5(conv4)
        
        pool = self.pool(conv5)
        center = self.center( pool )  
        dec5 = self.dec5(torch.cat([center, conv5], 1))        
        dec4 = self.dec4(torch.cat([dec5, conv4], 1))
        dec3 = self.dec3(torch.cat([dec4, conv3], 1))
        dec2 = self.dec2(torch.cat([dec3, conv2], 1))
        dec1 = self.dec1(dec2)           
        
        #attention map
        x = self.attention_map( dec1 ) 
        return x

找到了出错的位置，如何修改呢？参照资料3，首先将self.encoder中的参数的requires_grad都置为False，然后在优化器中加入过滤器，只更新requires_grad = True的参数。

for param in net.encoder.parameters():
    param.requires_grad = False

optimizer = optim.SGD(filter(lambda p: p.requires_grad, net.parameters()), lr=opt.lr, momentum=0.9, weight_decay=5e-4)

找了2天，终于找到错误的位置，太难了。