FCN的车牌图像识别，end-to-end 目标定位、图像识别

这里写图片描述

上图，分割结果和识别结果都非常好！！！

一、样本与label制作：效果如下图：
这里写图片描述

这里写图片描述

分别一一对应：

Fully Convolutional Networks forSemantic Segmentation

1.概览&主要贡献
提出了一种end-to-end的做semantic segmentation的方法，简称FCN。
如下图所示，直接拿segmentation 的 ground truth作为监督信息，训练一个端到端的网络，让网络做pixelwise的prediction，直接预测label map。
这里写图片描述

2.问题&解决办法

1）如何做pixelwise的prediction？

传统的网络是subsampling的，对应的输出尺寸会降低，要想做pixelwiseprediction，必须保证输出尺寸。
解决办法：
（1）对传统网络如AlexNet，VGG等的最后全连接层变成卷积层。
这里写图片描述

例如VGG16中第一个全连接层是25088x4096的，将之解释为512x7x7x4096的卷积核，则如果在一个更大的输入图像上进行卷积操作（上图的下半部分），原来输出4096维feature的节点处（上图的上半部分），就会输出一个coarsefeature map。
这样做的好处是，能够很好的利用已经训练好的supervisedpre-training的网络，不用像已有的方法那样，从头到尾训练，只需要fine-tuning即可，训练efficient。
（2）加In-network upsampling layer。
对中间得到的featuremap做bilinear上采样，就是反卷积层。实现把conv的前传和反传过程对调一下即可。
2）如何refine，得到更好的结果？

upsampling中步长是32，输入为3x500x500的时候，输出是544x544，边缘很不好，并且limit thescale of detail of the upsampling output。
解决办法：
这里写图片描述

采用skiplayer的方法，在浅层处减小upsampling的步长，得到的finelayer 和高层得到的coarselayer做融合，然后再upsampling得到输出。这种做法兼顾local和global信息，即文中说的combiningwhat and where，取得了不错的效果提升。FCN-32s为59.4，FCN-16s提升到了62.4，FCN-8s提升到62.7。可以看出效果还是很明显的。

3.训练细节

用AlexNet，VGG16或者GoogleNet训练好的模型做初始化，在这个基础上做fine-tuning，全部都fine-tuning。
采用wholeimage做训练，不进行patchwisesampling。实验证明直接用全图已经很effectiveand efficient。
对classscore的卷积层做全零初始化。随机初始化在性能和收敛上没有优势。
这里写图片描述

模型描述：

name: "FCN"

input: "data"
input_dim: 1
input_dim: 3
input_dim: 80
input_dim: 200
###########################################################
layer {
  name: "conv1_1"
  type: "Convolution"
  bottom: "data"
  top: "conv1_1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    pad: 24
    kernel_size: 3
    engine: CAFFE
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "relu1_1"
  type: "ReLU"
  bottom: "conv1_1"
  top: "conv1_1"
}
layer {
  name: "conv1_2"
  type: "Convolution"
  bottom: "conv1_1"
  top: "conv1_2"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    pad: 1
    kernel_size: 3
    engine: CAFFE
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "relu1_2"
  type: "ReLU"
  bottom: "conv1_2"
  top: "conv1_2"
}
layer {
  name: "pool1"
  type: "Pooling"
  bottom: "conv1_2"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "conv2_1"
  type: "Convolution"
  bottom: "pool1"
  top: "conv2_1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    pad: 1
    kernel_size: 3
    engine: CAFFE
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "relu2_1"
  type: "ReLU"
  bottom: "conv2_1"
  top: "conv2_1"
}
layer {
  name: "conv2_2"
  type: "Convolution"
  bottom: "conv2_1"
  top: "conv2_2"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    pad: 1
    kernel_size: 3
    engine: CAFFE
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "relu2_2"
  type: "ReLU"
  bottom: "conv2_2"
  top: "conv2_2"
}
layer {
  name: "pool2"
  type: "Pooling"
  bottom: "conv2_2"
  top: "pool2"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "conv3_1"
  type: "Convolution"
  bottom: "pool2"
  top: "conv3_1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 256
    pad: 1
    kernel_size: 3
    engine: CAFFE
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "relu3_1"
  type: "ReLU"
  bottom: "conv3_1"
  top: "conv3_1"
}
layer {
  name: "conv3_2"
  type: "Convolution"
  bottom: "conv3_1"
  top: "conv3_2"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 256
    pad: 1
    kernel_size: 3
    engine: CAFFE
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "relu3_2"
  type: "ReLU"
  bottom: "conv3_2"
  top: "conv3_2"
}
layer {
  name: "conv3_3"
  type: "Convolution"
  bottom: "conv3_2"
  top: "conv3_3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 256
    pad: 1
    kernel_size: 3
    engine: CAFFE
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "relu3_3"
  type: "ReLU"
  bottom: "conv3_3"
  top: "conv3_3"
}
layer {
  name: "pool3"
  type: "Pooling"
  bottom: "conv3_3"
  top: "pool3"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "score4"
  type: "Convolution"
  bottom: "pool3"
  top: "score4"
  param {
    lr_mult: 0.01
    decay_mult: 1
  }
  param {
    lr_mult: 0.02
    decay_mult: 0
  }
  convolution_param {
    num_output: 69
    kernel_size: 1
    engine: CAFFE
  }
}
layer {
  name: "upscore_new"
  type: "Deconvolution"
  bottom: "score4"
  top: "upscore_new"
  param {
    lr_mult: 0
  }
  convolution_param {
    num_output: 69
    bias_term: false
    kernel_size: 16
    stride: 8
  }
}
layer {
  type: "Crop"
  name: "score"
  top: "score"
  bottom: "upscore_new" 
  bottom: "data"
  crop_param{
    axis:2
    offset:19
    }
}
layer {
  type: "Softmax"
  name: "score1"
  top: "score1"
  bottom: "score" 
}

FCN的车牌图像识别，end-to-end 目标定位、图像识别

猜你喜欢