How to ignore log print a unused layer in caffe - machine-learning

I have an prototxt as follows:
layer {
name: "data"
type: "HDF5Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
hdf5_data_param {
source: "./train.txt"
batch_size: 2
}
}
layer {
name: "data_scale2"
type: "HDF5Data"
top: "data_scale2"
top: "label_scale2"
include {
phase: TRAIN
}
hdf5_data_param {
source: "./train_scale2.txt"
batch_size: 2
}
}
where the second layer contains the scale factor of 2 of the original data, I named it as data_scale2. During training, I just used the data,label and data_scale2 without using label_scale2. Hence, when I run training, the value of label_scale2 always printing in my terminal. How can I ignore this? This log looks like
I0302 18:01:57.356312 30995 solver.cpp:245] Train net output #221177: label_scale2 = 0
I0302 18:01:57.356314 30995 solver.cpp:245] Train net output #221178: label_scale2 = 0
I0302 18:01:57.356320 30995 solver.cpp:245] Train net output #221179: label_scale2 = 0
I0302 18:01:57.356324 30995 solver.cpp:245] Train net output #221180: label_scale2 = 0
I0302 18:01:57.356328 30995 solver.cpp:245] Train net output #221181: label_scale2 = 0
I0302 18:01:57.356329 30995 solver.cpp:245] Train net output #221182: label_scale2 = 0
I0302 18:01:57.356333 30995 solver.cpp:245] Train net output #221183: label_scale2 = 0

General solution:
If you wan to silence a "top" layer you can use "Silence" layer:
layer {
type: "Silence"
name: "silence_this_layer_for_me"
bottom: "label_scale2"
}
And that's it! You won;t hear from label_scale2 again
A solution for HDF5Data
Alternatively, you do not have to expose all datasets of an hdf5 input. You can simply comment out the redundant "top":
layer {
name: "data_scale2"
type: "HDF5Data"
top: "data_scale2"
# top: "label_scale2"
include {
phase: TRAIN
}
hdf5_data_param {
source: "./train_scale2.txt"
batch_size: 2
}
}

Related

Label smoothing in caffe with prototxt without data regeneration

I've got a huge data set in LMDB (40Gb) that I use for training a binary classifier with caffe.
Data layer in Caffe contains integer labels.
Are there any ready layers that could transform them into floats with adding some random jitter, so I could apply label smoothing technique, as described in 7.5.1 here
I have seen examples with HDF5, but they require regenerating data set, and I would like to avoid it.
You can use DummyData layer to generate the random noise you wish to add to the labels. Once you have the noise, use Eltwise layer to sum them up:
layer {
name: "noise"
type: "DummyData"
top: "noise"
dummy_data_param {
shape { dim: 10 dim: 1 dim: 1 dim: 1 } # assuming batch size = 10
data_filler { type: "uniform" min: -0.1 max: 0.1 } # noise ~U(-0.1, 0.1)
}
}
layer {
name: "label_noise"
type: "Eltwise"
bottom: "label" # the input integer labels
bottom: "noise"
top: "label_noise"
eltwise_param { operation: SUM }
}

I can't get Caffe working

After some struggling, I decided to try a most simple task, training a network to classify weither a number is non-negtive. And I failed...
I generated the data with following code. And I'm not sure if it is right. I read the data back from the file, and it looked right, though...
#pragma comment(lib, "hdf5")
#pragma comment(lib, "hdf5_cpp")
#include <cstdint>
#include <array>
#include <random>
#include <vector>
using namespace std;
#include <H5Cpp.h>
using namespace H5;
mt19937 rng;
float randf(float i_min, float i_max)
{
return rng() * ((i_max - i_min) / 0x100000000) + i_min;
}
#define NAME "pos_neg"
#define TRAIN_SET_SIZE 0x100000
#define TEST_SET_SIZE 0x10000
void make(const string &i_cat, uint32_t i_count)
{
H5File file(NAME "." + i_cat + ".h5", H5F_ACC_TRUNC);
hsize_t dataDim[2] = { i_count, 1 };
hsize_t labelDim = i_count;
FloatType dataType(PredType::NATIVE_FLOAT);
DataSpace dataSpace(2, dataDim);
DataSet dataSet = file.createDataSet("data", dataType, dataSpace);
IntType labelType(PredType::NATIVE_INT);
DataSpace labelSpace(1, &labelDim);
DataSet labelSet = file.createDataSet("label", labelType, labelSpace);
vector<float> data(i_count);
vector<int> labels(i_count);
for (uint32_t i = 0; i < i_count / 2; ++i)
{
labels[i * 2] = 0;
data[i * 2] = randf(0.f, 1.f);
labels[i * 2 + 1] = 1;
data[i * 2 + 1] = randf(-1.f, 0.f);
}
dataSet.write(&data[0], PredType::NATIVE_FLOAT);
labelSet.write(&labels[0], PredType::NATIVE_INT);
}
int main()
{
make("train", TRAIN_SET_SIZE);
make("test", TEST_SET_SIZE);
}
And the network looks like this
name: "PosNegNet"
layer {
name: "data"
type: "HDF5Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
hdf5_data_param {
source: "pos_neg_train.txt"
batch_size: 64
}
}
layer {
name: "data"
type: "HDF5Data"
top: "data"
top: "label"
include {
phase: TEST
}
hdf5_data_param {
source: "pos_neg_test.txt"
batch_size: 65536
}
}
layer {
name: "fc1"
type: "InnerProduct"
bottom: "data"
top: "fc1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "fc1"
bottom: "label"
top: "loss"
}
layer {
name: "accuracy"
type: "Accuracy"
bottom: "fc1"
bottom: "label"
top: "accuracy"
include {
phase: TEST
}
}
And and one set of parameters I tried
net: "pos_neg.prototxt"
test_iter: 1
test_interval: 500
base_lr: 0.001
momentum: 0.9
momentum2: 0.999
lr_policy: "fixed"
display: 100
max_iter: 10000
snapshot: 5000
snapshot_prefix: "pos_neg"
type: "Adam"
solver_mode: GPU
And I ran caffe.exe on Windows. And I always got loss = 0, accuracy = 0.5.
I know I must have done something wrong, but I don't know from where to look, well, other than digging up source code...
And I found that caffe is fairly slow. I got only around 16 iterations per second for a float[64] data with 1024 item per batch on a 1080Ti. Was it normal or I did something wrong again?
Set num_output: 2 in your "fc1": when using "SoftmaxWithLoss" and/or "Accuracy" layers caffe expects your prediction to be a vector of class probabilities. In your case, you have two classes, thus this vector should be of length 2 (and not 1 as it currently stands).
Alternatively, you can keep num_output: 1 and switch the loss to "SigmoidCrossEntropyLoss" layer. However, you will not be able to use "Accuracy" layer anymore...

error in making skip-layer connection network based on VGG16 in caffe

I am currently reading the paper on 'CMS-RCNN: Contextual Multi-Scale Region-based CNN for Unconstrained Face Detection', it is using the skip-connection to fuse conv3-3, conv4-3 and conv5-3 together, the steps are shown below
Extract the feature maps of the face region (at multiple scales conv3-3, conv4-3, conv5-3) and apply RoI-Pooling to it (i.e. convert to a fixed height and width).
L2-normalize each feature map.
Concatenate the (RoI-pooled and normalized) feature maps of the face (at multiple scales) with each other (creates one tensor).
Apply a 1x1 convolution to the face tensor.
Apply two fully connected layers to the face tensor, creating a vector.
I used the caffe and made a prototxt based on faster-RCNN VGG16 , the following parts are added into the original prototxt
# roi pooling the conv3-3 layer and L2 normalize it
layer {
name: "roi_pool3"
type: "ROIPooling"
bottom: "conv3_3"
bottom: "rois"
top: "pool3_roi"
roi_pooling_param {
pooled_w: 7
pooled_h: 7
spatial_scale: 0.25 # 1/4
}
}
layer {
name:"roi_pool3_l2norm"
type:"L2Norm"
bottom: "pool3_roi"
top:"pool3_roi"
}
-------------
# roi pooling the conv4-3 layer and L2 normalize it
layer {
name: "roi_pool4"
type: "ROIPooling"
bottom: "conv4_3"
bottom: "rois"
top: "pool4_roi"
roi_pooling_param {
pooled_w: 7
pooled_h: 7
spatial_scale: 0.125 # 1/8
}
}
layer {
name:"roi_pool4_l2norm"
type:"L2Norm"
bottom: "pool4_roi"
top:"pool4_roi"
}
--------------------------
# roi pooling the conv5-3 layer and L2 normalize it
layer {
name: "roi_pool5"
type: "ROIPooling"
bottom: "conv5_3"
bottom: "rois"
top: "pool5"
roi_pooling_param {
pooled_w: 7
pooled_h: 7
spatial_scale: 0.0625 # 1/16
}
}
layer {
name:"roi_pool5_l2norm"
type:"L2Norm"
bottom: "pool5"
top:"pool5"
}
# concat roi_pool3, roi_pool4, roi_pool5 and apply 1*1 conv
layer {
name:"roi_concat"
type: "Concat"
concat_param {
axis: 1
}
bottom: "pool5"
bottom: "pool4_roi"
bottom: "pool3_roi"
top:"roi_concat"
}
layer {
name:"roi_concat_1*1_conv"
type:"Convolution"
top:"roi_concat_1*1_conv"
bottom:"roi_concat"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 1
weight_filler{
type:"xavier"
}
bias_filler{
type:"constant"
}
}
}
layer {
name: "fc6"
type: "InnerProduct"
bottom: "roi_concat_1*1_conv"
top: "fc6"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 4096
}
}
during the training, I met such a issue
F0616 16:43:02.899025 3712 net.cpp:757] Cannot copy param 0 weights from layer 'fc6'; shape mismatch. Source param shape is 1 1 4096 25088 (102760448); target param shape is 4096 10368 (42467328).
To learn this layer's parameters from scratch rather than copying from a saved net, rename the layer.
I could find out what goes wrong, I need some help from you if you can spot some problem or explanation.
Really appreciated!!
The error message you got is quite clear. You are trying to fine-tune the weights of the layers, but for "fc6" layer you have a problem:
The original net you copied the weights from had "fc6" layer with input dimension of 10368. On the other hand, your "fc6" layer has input dimension of 25088. You cannot use the same W matrix (aka param 0 of this layer) if the input dimension is different.
Now that you know the problem, look at the error message again:
Cannot copy param 0 weights from layer 'fc6'; shape mismatch.
Source param shape is 1 1 4096 25088 (102760448);
target param shape is 4096 10368 (42467328).
Caffe cannot copy W matrix (param 0) of "fc6" layer, its shape does not match the shape of W stored in .caffemodel you are trying to fine tune.
What can you do?
Simply read the next line of the error message:
To learn this layer's parameters from scratch rather than copying from a saved net, rename the layer.
Just rename the layer, and caffe will learn the weights from scratch (only for this layer).

FCN:Check failed: outer_num_ * inner_num_ == bottom[1]->count()

I design a net the same as FCN.Input data is 1*224*224,Input label is 1*224*224.but I meet error:
F0502 07:57:30.032742 18127 softmax_loss_layer.cpp:47] Check failed: outer_num_ * inner_num_ == bottom[1]->count() (50176 vs. 1) Number of labels must match number of predictions; e.g., if softmax axis == 1 and prediction shape is (N, C, H, W), label count (number of labels) must be N*H*W, with integer values in {0, 1, ..., C-1}.
here is the input structure:
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
image_data_param {
ource: "/home/zhaimo/fcn-master/mo/train.txt"
batch_size: 1
shuffle: true
}
}
the softmax layers:
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "upscore1"
bottom: "label"
top: "loss"
loss_param {
ignore_label: 255
normalize: false
}
}
the train.txt file:
/home/zhaimo/fcn-master/data/vessel/train/original/01.png /home/zhaimo/SegNet/data/vessel/train/label/01.png
/home/zhaimo/fcn-master/data/vessel/train/original/02.png /home/zhaimo/SegNet/data/vessel/train/label/02.png
/home/zhaimo/fcn-master/data/vessel/train/original/03.png /home/zhaimo/SegNet/data/vessel/train/label/03.png
/home/zhaimo/fcn-master/data/vessel/train/original/04.png /home/zhaimo/SegNet/data/vessel/train/label/04.png
the first file name is input data and the second one is its label.
===========================update=======================================
I tried to use two ImageData layer as input:
layer {
name: "data"
type: "ImageData"
top: "data"
image_data_param {
source: "/home/zhaimo/fcn-master/mo/train_o.txt"
batch_size: 1
shuffle: false
}
}
layer {
name: "label"
type: "ImageData"
top: "label"
image_data_param {
source: "/home/zhaimo/fcn-master/mo/train_l.txt"
batch_size: 1
shuffle: false
}
}
but meet another error:
I0502 08:34:46.429774 19100 layer_factory.hpp:77] Creating layer data
I0502 08:34:46.429808 19100 net.cpp:100] Creating Layer data
I0502 08:34:46.429816 19100 net.cpp:408] data -> data
F0502 08:34:46.429834 19100 layer.hpp:389] Check failed: ExactNumTopBlobs() == top.size() (2 vs. 1) ImageData Layer produces 2 top blob(s) as output.
*** Check failure stack trace: ***
Aborted (core dumped)
train_o.txt:
/home/zhaimo/fcn-master/data/vessel/train/original/01.png
/home/zhaimo/fcn-master/data/vessel/train/original/02.png
/home/zhaimo/fcn-master/data/vessel/train/original/03.png
/home/zhaimo/fcn-master/data/vessel/train/original/04.png
/home/zhaimo/fcn-master/data/vessel/train/original/05.png
train_l.txt:
/home/zhaimo/SegNet/data/vessel/train/label/01.png
/home/zhaimo/SegNet/data/vessel/train/label/02.png
/home/zhaimo/SegNet/data/vessel/train/label/03.png
/home/zhaimo/SegNet/data/vessel/train/label/04.png
/home/zhaimo/SegNet/data/vessel/train/label/05.png
===============================Update2===================================
if I use two ImageData layers,how to modify the deploy.prototxt?
here is the file I wrote:
layer {
name: "data"
type: "ImageData"
top: "data"
top: "tmp0"
input_param { shape: { dim: 1 dim: 1 dim: 224 dim: 224 } }
}
and the forward.py file:
import numpy as np
from PIL import Image
caffe_root = '/home/zhaimo/'
import sys
sys.path.insert(0, caffe_root + 'caffe-master/python')
import caffe
# load image, switch to BGR, subtract mean, and make dims C x H x W for Caffe
im = Image.open('/home/zhaimo/fcn-master/data/vessel/test/13.png')
in_ = np.array(im, dtype=np.float32)
#in_ = in_[:,:,::-1]
#in_ -= np.array((104.00698793,116.66876762,122.67891434))
#in_ = in_.transpose((2,0,1))
# load net
net = caffe.Net('/home/zhaimo/fcn-master/mo/deploy.prototxt', '/home/zhaimo/fcn-master/mo/snapshot/train/_iter_200000.caffemodel', caffe.TEST)
# shape for input (data blob is N x C x H x W), set data
net.blobs['data'].reshape(1, *in_.shape)
net.blobs['data'].data[...] = in_
# run net and take argmax for prediction
net.forward()
out = net.blobs['score'].data[0].argmax(axis=0)
plt.axis('off')
plt.savefig('/home/zhaimo/fcn-master/mo/result/13.png')
but I meet the error:
F0504 08:16:46.423981 3383 layer.hpp:389] Check failed: ExactNumTopBlobs() == top.size() (2 vs. 1) ImageData Layer produces 2 top blob(s) as output.
how to modify the forward.py file,please?
Your problem is with the data top blob numbers. For two imagedata layer use this:
layer {
name: "data"
type: "ImageData"
top: "data"
top: "tmp"
image_data_param {
source: "/home/zhaimo/fcn-master/mo/train_o.txt"
batch_size: 1
shuffle: false
}
}
layer {
name: "label"
type: "ImageData"
top: "label"
top: "tmp1"
image_data_param {
// you probably also need
//is_color: false
source: "/home/zhaimo/fcn-master/mo/train_l.txt"
batch_size: 1
shuffle: false
}
}
In the text file just set all label to 0. You are not going to use tmp/tmp1 so it doesn't matter.

How to train Caffe with only G and B channels

Is there anyway to use only G and B channels for training Caffe using "ImageData" input layer?
You can add a convolution layer on top of your input that will select G and B:
layer {
name: "select_B_G"
type: "Convolution"
bottom: "data"
top: "select_B_G"
convolution_param { kernel_size: 1 num_output: 2 bias_term: false }
param { lr_mult: 0 } # do not learn parameters for this layer
}
You'll need to do some net surgery prior to training to set the weights for this layer to be
net.params['select_B_G'][0].data[...] = np.array( [[1,0,0],[0,1,0]], dtype='f4')
Note: sometimes images loaded to caffe are going through channel-swap transformation, i.e., RGB -> BGR, therefore you need to be careful what channels you pick.
I wrote a simple python layer to do this, by the way, I did't test this code.
import caffe
class ExtractGBChannelLayer(caffe.Layer):
def setup(self,bottom,top):
pass
def reshape(self,bottom,top):
bottom_shape=bottom[0].data.shape
top_shape=[bottom_shape[0],2,bottom_shape[2],bottom_shape[3]] #because we only want G and B channels.
top[0].reshape(*top_shape)
def forward(self,bottom,top):
#copy G and B channel to top, note caffe BGR order!
top[0].data[:,0,...]=bottom[0].data[:,1,...]
top[0].data[:, 1, ...] = bottom[0].data[:, 0, ...]
def backward(self,top,propagate_down,bottom):
pass
You can save this file as MyPythonLayer.py
In you prototxt you can insert a layer after ImageDataLayer like this
layer {
name: "GB"
type: "Python"
bottom: "data"
top: "GB"
python_param {
module: "MyPythonLayer"
layer: "ExtractGBChannelLayer"
}
}
Hope this works fine.
This is the Matlab code I used and it works.
caffe.reset_all(); % reset caffe
caffe.set_mode_gpu();
gpu_id = 0; % we will use the first gpu in this demo
caffe.set_device(gpu_id);
net_model = ['net_images.prototxt'];
net = caffe.Net(net_model, 'train')
a = zeros(1,1,3,2);
a(1,1,:,:) = [[1,0,0];[0,1,0]]'; % caffe uses BGR color channel order
net.layers('select_B_G').params(1).set_data(a);
solver = caffe.Solver(solverFN);
solver.solve();
net.save(fullfile(model_dir, 'my_net.caffemodel'));

Resources