Paper address: https://arxiv.org/pdf/1512.03385.pdf
pytorch official pre-training model address:
'resnet18': 'https://download.pytorch.org/models/resnet18-f37072fd.pth',
'resnet34': 'https://download.pytorch.org/models/resnet34-b627a593.pth',
'resnet50': 'https://download.pytorch.org/models/resnet50-0676ba61.pth',
'resnet101': 'https://download.pytorch.org/models/resnet101-63fe2227.pth',
'resnet152': 'https://download.pytorch.org/models/resnet152-394f9c45.pth',
'resnext50_32x4d': 'https://download.pytorch.org/models/resnext50_32x4d-7cdf4587.pth',
'resnext101_32x8d': 'https://download.pytorch.org/models/resnext101_32x8d-8ba56ff5.pth',
'wide_resnet50_2': 'https://download.pytorch.org/models/wide_resnet50_2-95faca4d.pth',
'wide_resnet101_2': 'https://download.pytorch.org/models/wide_resnet101_2-32ee1156.pth'pytorch official resnet network code (including resnet18, 34, 50, 101, 152, resnext50_32x4d, resnext101_32x8d, wide_resnet50_2, wide_resnet101_2): torchvision.models.resnet — Torchvision 0.11.0 documentation 
https://pytorch.org/vision/stable/ _modules /torchvision/models/resnet.html
-------------------------------------------------- --------------Dividing line---------------------------------- ---------------------
As shown in the figure below, several common resnet network structures are introduced in the paper. In order to understand the principle and code implementation of resnet, it is enough to analyze the structure and reproduce the code of resnet18 and resnet50. The block of resnet18 is BasicBlock when it is implemented in the code, and the block of resnet50 is Bottleneck when it is implemented. The difference between BasicBlock and Bottleneck is that the former is composed of two 3x3 convolutions, and the latter is composed of two 1x1 convolutions plus a 3x3 convolution.
The following figure is the network model structure diagram of resnet18 and resnet50 , transferred from
Some points to note:
1. The first shortcut in the first layer (composed of multiple blocks) of resnet50 uses 1x1 convolution, but the stride is 1, and the size of the feature map remains unchanged;
2. The first shortcut of each layer uses 1x1 convolution to increase the output dimension of the previous block (two 1x1 convolutions + one 3x3 convolution to form a block, and the stride is 2 from the second layer, and the feature map is long. The width size is reduced by half to keep the same output size as the previous layer;
3. The stride of each convolution in the first layer of resnet50 is 1, the stride of the 3x3 convolution in the first block from the second layer is 2, and the rest are all 1;
pytorch implementation of BasicBlock and Bottleneck:
BasicBlock
class BasicBlock(nn.Module):
expansion: int = 1
def __init__(
self,
inplanes: int,
planes: int,
stride: int = 1,
downsample: Optional[nn.Module] = None,
groups: int = 1,
base_width: int = 64,
dilation: int = 1,
norm_layer: Optional[Callable[..., nn.Module]] = None
) -> None:
super(BasicBlock, self).__init__()
if norm_layer is None:
norm_layer = nn.BatchNorm2d
if groups != 1 or base_width != 64:
raise ValueError('BasicBlock only supports groups=1 and base_width=64')
if dilation > 1:
raise NotImplementedError("Dilation > 1 not supported in BasicBlock")
# Both self.conv1 and self.downsample layers downsample the input when stride != 1
self.conv1 = conv3x3(inplanes, planes, stride)
self.bn1 = norm_layer(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = conv3x3(planes, planes)
self.bn2 = norm_layer(planes)
self.downsample = downsample
self.stride = stride
def forward(self, x: Tensor) -> Tensor:
identity = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
if self.downsample is not None:
identity = self.downsample(x)
out += identity
out = self.relu(out)
return outBottleneck
class Bottleneck(nn.Module):
# Bottleneck in torchvision places the stride for downsampling at 3x3 convolution(self.conv2)
# while original implementation places the stride at the first 1x1 convolution(self.conv1)
# according to "Deep residual learning for image recognition"https://arxiv.org/abs/1512.03385.
# This variant is also known as ResNet V1.5 and improves accuracy according to
# https://ngc.nvidia.com/catalog/model-scripts/nvidia:resnet_50_v1_5_for_pytorch.
expansion: int = 4
def __init__(
self,
inplanes: int,
planes: int,
stride: int = 1,
downsample: Optional[nn.Module] = None,
groups: int = 1,
base_width: int = 64,
dilation: int = 1,
norm_layer: Optional[Callable[..., nn.Module]] = None
) -> None:
super(Bottleneck, self).__init__()
if norm_layer is None:
norm_layer = nn.BatchNorm2d
width = int(planes * (base_width / 64.)) * groups
# Both self.conv2 and self.downsample layers downsample the input when stride != 1
self.conv1 = conv1x1(inplanes, width)
self.bn1 = norm_layer(width)
self.conv2 = conv3x3(width, width, stride, groups, dilation)
self.bn2 = norm_layer(width)
self.conv3 = conv1x1(width, planes * self.expansion)
self.bn3 = norm_layer(planes * self.expansion)
self.relu = nn.ReLU(inplace=True)
self.downsample = downsample
self.stride = stride
def forward(self, x: Tensor) -> Tensor:
identity = x
out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)
out = self.conv2(out)
out = self.bn2(out)
out = self.relu(out)
out = self.conv3(out)
out = self.bn3(out)
if self.downsample is not None:
identity = self.downsample(x)
out += identity
out = self.relu(out)
return out
Welcome everyone to join the group exchange:
