版权声明:未经博主允许,不得转载! https://blog.csdn.net/u011681952/article/details/86157274
InnerProduct Layer是全连接层,CNN中常常出现在分类网络的末尾,将卷积全连接化或分类输出结果,当然它的用处很多,不只是分类网络中
首先我们先看一下 InnerProductParameter
message InnerProductParameter {
optional uint32 num_output = 1; // The number of outputs for the layer //输出的个数
optional bool bias_term = 2 [default = true]; // whether to have bias terms //是否使用bias
optional FillerParameter weight_filler = 3; // The filler for the weight //全连接权重
optional FillerParameter bias_filler = 4; // The filler for the bias //bias权重
// The first axis to be lumped into a single inner product computation;
// all preceding axes are retained in the output.
// May be negative to index from the end (e.g., -1 for the last axis).
//将第一个轴集中进行单个内积计算
optional int32 axis = 5 [default = 1];
// Specify whether to transpose the weight matrix or not.
// If transpose == true, any operations will be performed on the transpose
// of the weight matrix. The weight matrix itself is not going to be transposed
// but rather the transfer flag of operations will be toggled accordingly.
// 是否要转置权矩阵。如果true,则对权矩阵执行转置操作,注意权重矩阵本身不会被置换,而是操作传输标志作相应切换
optional bool transpose = 6 [default = false];
}
InnerProduct layer 在prototxt里面的书写:
layer {
name: "fc"
type: "InnerProduct"
bottom: "fc_in"
top: "fc_out"
param { # 权重学习参数
lr_mult: 1 # 学习率
decay_mult: 1
}
param { # bias 学习参数
lr_mult: 2 # 一般情况,bias 学习率是权重学习率的两倍.
decay_mult: 0
}
inner_product_param {
num_output: 1000 # 输出单元个数
weight_filler { # 权重初始化方法
type: "gaussian"
std: 0.005
}
bias_filler { # bias 初始化方法
type: "constant"
value: 0.1
}
}
}
有些网络在实现时,常常用卷积来代替全连接层的功能,如MobileNet里面:
layer {
name: "fc7"
type: "Convolution"
bottom: "pool6"
top: "fc7"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 1000
kernel_size: 1
weight_filler {
type: "msra"
}
bias_filler {
type: "constant"
value: 0
}
}
}