How to use C++ to extract features from any image (read data from memory)

Regarding the use of C++ interface to extract features, caffe officially provides a routine of extract_features.cpp. However, the input of this file is blob data. Even if the input layer USES ImageData, it needs to specify the location of the image in deploy.prototxt, which is very inconvenient.

If you want to read an image using opencv and extract features using caffe trained model, you need to rewrite the input layer. In addition, the default output of the official routine is in leveldb format, and we can also get multi-dimensional features (arrays) of type float, which is more flexible to integrate into our project.

01

First we need to rewrite deploy. The input layer of prototxt is "MemoryData" :

layer {
 name: "data"
 type: "MemoryData"
 top: "data"
 top: "label"
 memory_data_param{
 batch_size:1
 channels:3
 height:100
 width:100
 }
}

"ImageData", "Data" and so on May have been used in the previous training, but the change to "MemoryData" does not affect it.

02

I'm ready to extract the name of the layer is "res5_6", is "InnerProduct" 1 layer before, when I want to extract the "InnerProduct" the output of the whole connection layer, is always an error, prompt initial parameters and network parameters does not match (that is, trained model and now deploy network dimension is not 1), so I had to extract 1 layer before, and to connect the whole layer block, shield method is the name of the corresponding level in the prototxt break is good (relative to the caffemodel name). [the above problems have not been solved for the time being.

03

The following is the code of extract_features.cpp after the change:

#include <stdio.h> 
#include <string>
#include <vector>
#include <iostream>
#include <opencv2/opencv.hpp>
#include "boost/algorithm/string.hpp"
#include "google/protobuf/text_format.h"
#include "caffe/blob.hpp"
#include "caffe/common.hpp"
#include "caffe/net.hpp"
#include "caffe/proto/caffe.pb.h"
#include "caffe/util/io.hpp"
#include "caffe/layers/memory_data_layer.hpp"
#define NetTy float 
using namespace caffe;
using std::cout;
using std::endl;
using std::string;
/*  Load model function  */
template <typename Dtype>
caffe::Net<Dtype>* loadNet(std::string param_file, std::string pretrained_param_file, caffe::Phase phase)
{
 caffe::Net<Dtype>* net(new caffe::Net<Dtype>(param_file, phase));
 net->CopyTrainedLayersFrom(pretrained_param_file);
 return net;
}
int main()
{
 cv::Mat src;
 src = cv::imread("face_example/test.jpg"); //  Read test picture 
 cv::resize(src, src, cv::Size(100, 100)); //  I'm going to put a picture here resize to prototxt The input layer inside the specified size 
 caffe::Net<NetTy>* _net = loadNet<NetTy>("face_example/face_deploy.prototxt", "face_example/face.caffemodel", caffe::TEST); //  Load the network definition file and the parameter model 
 caffe::MemoryDataLayer<NetTy> *m_layer = (caffe::MemoryDataLayer<NetTy> *)_net->layers()[0].get(); //  Define an in-memory data layer pointer 
 std::vector<cv::Mat> dv = { src }; // AddMatVector(const vector<cv::Mat>& mat_vector,const vector<int>& labels)
 std::vector<int> label = { 0 }; // -------------------------------------------------------------------------
 m_layer->AddMatVector(dv, label); //  Add pictures and tags to  MemoryData layer 
 std::vector<caffe::Blob<NetTy>*> input_vec; //  Nonsense for function arguments 
 _net->Forward(input_vec);     //  perform 1 Second forward calculation 
 boost::shared_ptr<caffe::Blob<NetTy>> layerData = _net->blob_by_name("res5_6"); //  Gets the output of the specified layer 
 const NetTy* pstart = layerData->cpu_data(); // res5_6->cpu_data() It's a multi-dimensional data (array) 
 /*----- The output characteristics -----*/
 for (int i = 0; i < 30000; i++)
 {
  std::cout << *pstart << endl;
  pstart++;
 }
 return 0;
}