利用cpp提取caffe特征

写在前面

由于项目需要,利用caffe的cpp接口提取隐含层特征。

这里记录踩过的坑。

最终基于python与基于classification.cpp提取的特征保持一致,相关完整代码已整合github

opencv与caffe

一切的起源在于,利用opencv的dnn模块读取caffe模型,最终提取特征与利用caffe的python接口提取的特征不同。

opencv接口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#include <opencv2/dnn/dnn.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/core/utils/trace.hpp>
using namespace cv;
using namespace cv::dnn;


CV_TRACE_FUNCTION();
String modelTxt = "ResNet11.prototxt";
String modelBin = "ResNet11.caffemodel";
String probfile = "./jpegFile/prob.jpg";
Net net = dnn::readNetFromCaffe(modelTxt, modelBin);
Mat img = imread(imgFile.c_str());
Mat inputBlob = blobFromImage(img, 1, Size(96, 96),Scalar(119.935, 119.935, 119.935));
CV_TRACE_REGION("forward");
net.setInput(inputBlob, "data");
Mat prob = net.forward("fc1");

python接口

1.caffe.io.load_image()读取图片为RGB格式,而caffenet输入图片为BGR格式。采用opencvcv::imread()读取的图片即为BGR格式。

2.由于python通过np.load()读取均值文件,因而在prototxt中应剔除mean_file相关配置,这就是踩的一个大坑..

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
import numpy as np
import sys
caffe_root='/home/user/caffe/'
sys.path.insert(0,caffe_root+'python')
import caffe
import os
# 初始化并加载caffe模型
caffe.set_mode_cpu()
model_def = '/home/user/extract_feature/py_deploy.prototxt'
model_weights='/home/user/extract_feature/ResNet11.caffemodel'
net = caffe.Net(model_def, # defines the structure of the model
model_weights, # contains the trained weights
caffe.TEST) # use test mode (e.g., don't perform dropout)
mu=np.load('/home/user/extract_feature/mean.npy') #读取均值文件
mu = mu.mean(1).mean(1)
print 'mean-subtracted values:', zip('BGR', mu)


# caffe输入图片格式为BGR,因此需要对输入数据进行预处理操作
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_transpose('data', (2,0,1)) # move image channels to outermost dimension
transformer.set_mean('data', mu) # subtract the dataset-mean value in each channel
transformer.set_raw_scale('data', 255) # rescale from [0, 1] to [0, 255]
transformer.set_channel_swap('data', (2,1,0)) # swap channels from RGB to BGR
# set the size of the input (we can skip this if we're happy
# with the default; we can also change it later, e.g., for different batch sizes)
net.blobs['data'].reshape(3, 3, 96, 96)
image = caffe.io.load_image('/home/user/extract_feature/img/img_3.jpg')
transformed_image = transformer.preprocess('data', image)
net.blobs['data'].data[...] = transformed_image
### perform classification
output = net.forward()
feature=net.blobs['fc1'].data[0]
filehandle=open('py_out.txt','a')
filehandle.write(' '.join(str(a) for a in feature))
filehandle.write('\n')
filehandle.close

CPP相关API接口

参考链接examples/cpp_classification/classification.cpptools/extract_feature.cpp

初始化网络

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
#include "caffe/caffe.hpp"
#include <string>

//其他头文件
#include <algorithm>
#include <iosfwd>
#include <memory>
#include <utility>
#include <vector>

using namespace caffe;
string model_file="deploy.prototxt";
string model_weight="resnet.caffemodel";
string mean_file="mean.binaryproto";
//选择计算方式
#ifdef CPU_ONLY
Caffe::set_mode(Caffe::CPU);
#else
Caffe::set_mode(Caffe::GPU);
//初始化网络
shared_ptr<Net<float> > net.reset(new Net<float>(model_file,TEST));
//加载网络参数
net->CopyTrainedLayersFrom(model_weight);

读取图像均值

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
BlobProto blob_proto;
ReadProtoFromBinaryFileOrDie(mean_file.c_str(), &blob_proto);

//将BlobProto转化为Blob<float>
Blob<float> mean_blob;
mean_blob.FromProto(blob_proto);
//将均值文件转化为cv::Mat格式
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
std::vector<cv::Mat> channels;
float* data = mean_blob.mutable_cpu_data();
//按通道将mean文件单独抽取
for (int i = 0; i < num_channels_; ++i) {
cv::Mat channel(mean_blob.height(), mean_blob.width(), CV_32FC1, data);
channels.push_back(channel);
data += mean_blob.height() * mean_blob.width();
}

cv::Mat mean;
cv::merge(channels, mean);
cv::Scalar channel_mean = cv::mean(mean);
//最终获得Mat形式mean
cv::Mat mean_ = cv::Mat(input_geometry_, mean.type(), channel_mean);

利用opencv读取图像并进行预处理

其中的input_channels需要WrapInputLayer处理,具体函数功能没有弄清,先摆出来。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
void Classifier::WrapInputLayer(std::vector<cv::Mat>* input_channels) {
Blob<float>* input_layer = net_->input_blobs()[0];

int width = input_layer->width();
int height = input_layer->height();
float* input_data = input_layer->mutable_cpu_data();
for (int i = 0; i < input_layer->channels(); ++i) {
cv::Mat channel(height, width, CV_32FC1, input_data);
input_channels->push_back(channel);
input_data += width * height;
}
}



string file="lena.jpg";
cv::Mat img=cv::imread(file,-1);
//区分灰度图像,尺寸信息等略过
//减去均值文件
cv::Mat sample_normalized;
cv::substract(img,mean_,sample_normalized);
cv::split(sample_normalized, *input_channels);

根据名称提取对应层特征

1
2
3
4
5
6
7
8
9
10
11
12
Blob<float>* input_layer = net_->input_blobs()[0];
input_layer->Reshape(1, num_channels_,
input_geometry_.height, input_geometry_.width);
net->Reshape();
std::vector<cv::Mat> input_channels;
WrapInputLayer(&input_channels);
net->Forward();
//根据名称提取对应层特征
const shared_ptr<Blob<float> >output_layer = net_->blob_by_name("fc1");
const float* begin = output_layer->cpu_data();
const float* end = begin + output_layer->channels();
std::vector<float> feature_out = new std::vector<float>(begin, end);

cpp提取caffe特征

caffe自带两个例程可以用来提取某一层的特征。

examples/cpp_classification/classification.cpp原本是一个用来输出最后一层分类结果的例程。

tools/extract_feature.cpp读取lmdb数据或图片数据,提取所需层特征,并保存为lmdb格式。

lmdb读取

目前搜索结果,读取lmdb只有采用python的lmdb模块。相关代码参考1代码参考2

但测试发现,解析的特征与例程相距甚远。

相关解释为python中lmdb数据存放以key值按照字母排序整理,而caffe中似乎按照value大小进行整理。

本文标题:利用cpp提取caffe特征

文章作者:Lumo Wang

发布时间:2018年07月04日 - 20:07

最后更新:2018年11月07日 - 15:11

原始链接:https://luameows.github.io/2018/07/04/程序开发-利用cpp提取caffe特征/

许可协议: 署名-非商业性使用-禁止演绎 4.0 国际 转载请保留原文链接及作者。

请我喝杯咖啡,我会继续熬夜的~